The document discusses randomized controlled trials (RCTs). It summarizes that HSRU is funded by the Scottish Government Health Directorates, and the author accepts full responsibility for the talk. RCTs are the gold standard for evaluating healthcare interventions through random allocation of participants to experimental or control groups in order to minimize bias.

1. HSRU is funded by the Chief Scientist Office of the Scottish
Government Health Directorates . The author accepts full
responsibility for this talk.
Health Services Research Unit
University of Aberdeen
Randomised ControlledRandomised Controlled
Trials (RCTs)Trials (RCTs)
Graeme MacLennan

2. Health Services Research Unit
James LindJames Lind
• Born Edinburgh 1716
• On HMS Salisbury in 1747 he
allocated 12 men with scurvy
– Cider
– Seawater
– Horseradish, mustard, garlic
– Nutmeg
– Elixir Vitriol
– Oranges and Limes

3. Health Services Research Unit
Think about…Think about…
• Consider how you would go about evaluating the
following interventions
– Surgical versus medical termination of pregnancy
– Referral guidelines for radiographic examination
– Paracetamol and/or ibuprofen for treating children
with fever
– Nurse counsellors as an alternative to clinical
geneticists for genetic counselling
– Single dose of chemotherapy versus radiotherapy
treating testicular cancer
http://news.bbc.co.uk/1/hi/health/7647007.stm
– Cervical cancer vaccine
http://news.bbc.co.uk/1/hi/health/6223000.stm

4. Health Services Research Unit
The need to evaluate healthThe need to evaluate health
carecare
• Variations in health care
• Unproven treatments
• Inadequacies in care
• Inaccurate medical models
• Limitation of resources
• New innovations
• …
Crombie (1996)

5. Health Services Research Unit
Evaluation processEvaluation process
• Define research question
• What is already known?
• Identify appropriate study design
• Define population, intervention and criteria for
evaluation
• How large a study?
• Consider measurement of evaluation criteria
(“outcomes”)
– How often?
– Timing? Length of follow up?
– To whom? Who collects the data? What format?
• Analysis of data
• Dissemination and implementation

6. Health Services Research Unit
Define research question andDefine research question and
what is already knownwhat is already known
• Research question (PICOT)
– Population
– Intervention
– Control/comparator
– Outcome
– Target
• Has the question already been answered?
– Conduct review to assess what is know
about intervention

7. Health Services Research Unit
Definition of population,Definition of population,
intervention and “outcomes”intervention and “outcomes”
• Population
– Strict definition (explanatory) or flexible
(pragmatic)
• Intervention
– Dose of drug, timing etc
• “Outcomes”
– Health related Quality of Life
– Biochemical outcomes
– Symptoms
– Physical assessment
– Patient satisfaction
– Acceptability
– Cost-effectiveness

8. Health Services Research Unit
Measuring “outcome”Measuring “outcome”
• Questionnaires, interview, medical
notes etc
• Timing of questionnaires?
– Baseline (prior to treatment)
– Short term outcomes
– Long term outcomes
• Who collects the data?

9. Health Services Research Unit
Sources of systematicSources of systematic
errorserrors
• Selection bias
– can be introduced by the way in which comparison
groups are assembled
• Attrition bias
– systematic differences in withdrawal/follow up
• Performance bias
– Systematic differences in care provided
• Observation/detection bias
– systematic differences in observation,
measurement, assessment

10. Health Services Research Unit
What is a randomised controlled trial?What is a randomised controlled trial?
Simple Definition
• A study in which people are allocated
at random to receive one of several
interventions
(simple but powerful research tool)

11. Health Services Research Unit
Simple RCT modelSimple RCT model
Trial participantsTrial participants
RANDOMLYRANDOMLY allocatedallocated
to experimentalto experimental
oror CONTROLCONTROL groupgroup
CONTROLCONTROLEXPERIMENTEXPERIMENT

12. Health Services Research Unit
What is aWhat is a randomisedrandomised controlled trial?controlled trial?
• Random allocation to intervention
groups
• all participants have equal chance of
being allocated to each intervention
group
why RCTs are referred to as
randomised controlled trials

13. Health Services Research Unit
TerminologyTerminology
• Interventions are comparative regimes within
a trial
• Prophylactic, diagnostic, therapeutic e.g.
– preventative strategies
– screening programmes
– diagnostic tests
– drugs
– surgical techniques

14. Health Services Research Unit
What is a randomisedWhat is a randomised
controlledcontrolled trial?trial?
• One intervention is regarded as
control treatment (the group of
participants who receive this are the
control group)
• NOTE: Contemporaneous (not
historical controls)
why RCTs are referred to as randomised
controlled trials

15. Health Services Research Unit
TerminologyTerminology
Control Group
• can be
– conventional
practice
– no intervention
(this may be
conventional
practice)
– placebo
Experimental
Group
• receive new
intervention
• (also called
treatment group or
intervention group
interchangeably)

16. Health Services Research Unit
What is aWhat is a randomisedrandomised
controlled trialcontrolled trial ??
• RCTs are
– Experiments: investigators can influence
number, type, regime of interventions
– Quantitative: measure events rather than
try to interpret them in their natural
settings
– Comparative: compare two or more
interventions

17. Health Services Research Unit
What is a randomisedWhat is a randomised
controlled trial?controlled trial?
More Complex Definition
• Quantitative, comparative, controlled
experiments in which a group of
investigators study two or more
interventions in a series of participants
who are allocated randomly to each
intervention group

18. Health Services Research Unit
Inclusion/exclusion criteriaInclusion/exclusion criteria
• Decision rules applied to potential trial
participants to judge eligibility for
inclusion in trial
• See CONSORT statement
www.consort-statement.org
• Important that they are applied
identically to all groups in a trial!

19. Health Services Research Unit
What is randomisation?What is randomisation?
• Randomisation is the process of random
allocation
• Allocation is not determined by investigators,
clinicians or participants
• Equal chance of being assigned to each
intervention group
• Individual people
– patients
– caregivers (physicians, nurses etc)
• Groups of people, ‘cluster randomisation’
• (Covered in more depth in later lecture)

20. Health Services Research Unit
Pseudo-randomisationPseudo-randomisation
• Other allocation methods include
– according to date of birth
– the number on hospital records
– date of invitation etc.
• These are NOT regarded as random
• These are called pseudo- or quasi-
random

21. Health Services Research Unit
TerminologyTerminology
• Controlled clinical trials (CCTs) are
not the same as randomised controlled
trials
• Controlled clinical trials include non-
randomised controlled trials and
randomised controlled trials

22. Health Services Research Unit
Why use randomisation?Why use randomisation?
• Characteristics similar across groups at
baseline
• can isolate and quantify impact of
interventions with effects from other
factors minimised
• Risk of imbalance not abolished
completely even if perfect randomisation
• To combat selection bias
Unpredictability paradox: review of empirical comparisons of randomised and non-randomised
clinical trials, Kunz and Oxman 1998 BMJ
http://www.bmj.com/cgi/content/abstract/317/7167/1185

23. Health Services Research Unit
Why do we need a control group?Why do we need a control group?
• Don’t need a control group if
completely predictable results
– Parachutes when jumping from
plane
– New drug cures a few rabies cases
• But
– No intervention has 100% efficacy
– Many diseases recover
spontaneously

24. Health Services Research Unit
Regression to the meanRegression to the mean
• Occurs when an intervention aimed at
a group or characteristic that is very
different from average
• For example selecting people because
they have high blood pressure then
measuring them in future will see the
blood pressure measurements closer
to the mean of the population
Morton and Torgerson BMJ 2003 326:1083-4
Bland and Altman BMJ 1994 308:1499 and 309:780

25. Health Services Research Unit
DISTRIBUTION OFDISTRIBUTION OF
RESULTSRESULTS
threshold
measurement 1measurement 2

26. Health Services Research Unit
Hawthorne EffectHawthorne Effect
• Experimental effect in the direction
expected but not for the reason
expected
• Essentially studying/measuring
something can change what you
studying/measuring

27. Health Services Research Unit
Placebo EffectPlacebo Effect
• Effect (usually, but not always positive)
attributed to the expectation that a therapy
will have an effect
• The effect is due to the power of suggestion
• A placebo is an inert medication or
procedure
Waber et al 2008 JAMA Commercial Features of Placebo and
Therapeutic Efficacy
http://jama.ama-assn.org/cgi/content/full/299/9/1016

28. Effect size
Experimental group Control group
Hawthorne E.
R. mean
Placebo E.
Therapeutic E.
Real difference
Noise
Signal
EFFECT OF ANEFFECT OF AN
INTERVENTIONINTERVENTION

29. Health Services Research Unit
Minimising bias in RCTsMinimising bias in RCTs
• Blinding
– Single blind – participants are unaware of
treatment allocation
– Double blind – both participants and
investigators are unaware of treatment
allocation
– Requires use of placebos in drug trials
Schulz and Grimes (2002)

30. Health Services Research Unit
Concealment of random allocation listConcealment of random allocation list
• “Trials with inadequate allocation
concealment have been associated with
larger treatment effects compared with
trials in which authors reported adequate
allocation concealment”
Schulz KF (1995). Subverting randomisation in
controlled trials. JAMA, 274, 1456-8

31. Health Services Research Unit
Blinding, placebosBlinding, placebos
• RCTs should use the maximum degree of
blinding that is possible
• Placebo is a ‘dummy’ treatment given
when there is no obvious standard
treatment
– needed as the act of taking a treatment
may have some effect -need to
attribute
– double blind treatments must be
indistinguishable to those affected

32. Health Services Research Unit
Empirical evidence ofEmpirical evidence of
biasbias
MethodologicalIssue Increaseintreatment
effect(OR)
Inadequateconcealment of
treatmentallocation
41%
Unclearmethodofconcealment
oftreatmentallocation
30%
Trialnotblinded(whencould
havebeen)
17%
SchulzKFetal.JAMA1995;273:408-412

33. Health Services Research Unit
‘‘Explanatory’ and ‘Pragmatic’Explanatory’ and ‘Pragmatic’
questionsquestions
Explanatory
• Can it work in an
ideal setting …..?
• Efficacy
• Hypothesis testing
• Placebo controlled
• Double blind
Pragmatic
• Does it work in the real
world …..?
• Effectiveness
• Choice between
alternative approaches
to health care
• Standard care
• Open

34. Health Services Research Unit
Key differences betweenKey differences between
explanatory and pragmatic trialsexplanatory and pragmatic trials
(1)(1)
Explanatory Pragmatic
Question efficacy effectiveness
Setting ‘laboratory’ normal practice
Participants strictly defined broader, clinically indicated
(uncertainty)
Interventions strictly defined as clinical practice

35. Health Services Research Unit
Key differences betweenKey differences between
explanatory and pragmatic trialsexplanatory and pragmatic trials
(2)(2)
Explanatory Pragmatic
Outcomes short-term long-term, patient-
surrogates centered and
resource orientated
Size small (usually larger
single centre) (often multi-centre)
Analysis treatment received intention to treat
Relevance indirect direct
to practice

36. Health Services Research Unit
Example of selection biasExample of selection bias
for PP in an open trialfor PP in an open trial
Worse prognosisWorse prognosis
ExEx
pp
CtrlCtrl
None
None
White(2005)
E
E

37. Health Services Research Unit
Terminology: explanatoryTerminology: explanatory
versus pragmaticversus pragmatic
• Explanatory trials
– estimates efficacy - that is the benefit the
treatment produces under ideal conditions
• Pragmatic trials
– estimates effectiveness - that is the
benefit the treatment produces under
routine clinical practice
Roland M, Torgerson D. What are pragmatic trials? BMJ
1998;316:285

38. Health Services Research Unit
RCT as the GoldRCT as the Gold
StandardStandard
• The randomised controlled trial is
widely regarded as the gold standard
for evaluating health care technologies
because it allows us to be confident
that a difference in outcome can be
directly attributed to a difference in
the treatments and not due to some
other factor

39. Health Services Research Unit
RCT strengthsRCT strengths
• Confounding variables minimised
• Only research design which can in
principle yield causal relationships
– can clarify the direction of cause
and effect
• Accepted by EBM school
• Don’t have to know everything about
the participants

40. Health Services Research Unit
RCT limitationsRCT limitations
• Contamination of intervention groups
• Comparable controls
• Problems with blinding
• What to do about attrition?
• Are patients/professionals willing to be
in trial different from ‘refusers’?-
external validity
• Cost!

41. Health Services Research Unit
Other issues in RCTs (1)Other issues in RCTs (1)
• Ethics
• Management issues
• Interim analysis and ‘stopping rules’
– part of ethical concern
– mechanisms to avoid patient harm
– Data Monitoring and Safety
Committee required for trials
Clemens F et al Data monitoring in randomised controlled trials: surveys of recent
practice and policies. Clin Trials 2005;2(1):22-33.

42. Health Services Research Unit
Other issues in RCTs (2)Other issues in RCTs (2)
• A power calculation is essential for the
validity of a trial and will always be
necessary for grant applications and in
publications of the trial (later lecture)
• The methods of randomisation should
always be reported. It is not enough to
say that the patients were randomly
allocated to the treatments. (see
CONSORT)

43. Parallel group (simple) RCTParallel group (simple) RCT
design in practicedesign in practice
Patient eligible for either treatment
Patient gives informed consent
Yes No
Randomise Exclude from trial
Experimental
treatment
Standard
treatment
Standard
treatment

44. Health Services Research Unit
SummarySummary
• “Gold standard” of research designs
• Individual patients are randomly allocated to receive
the experimental treatment (intervention group) or the
standard treatment (control group)
• Maximises the potential for attribution
• Randomisation guards against selection bias between
the two treatment groups
• Standard statistical analysis
• Good internal validity
• May lack generalisability due to highly selected
participants
• Can be costly to set up and conduct, ethical issues

45. Health Services Research Unit
Good study designGood study design
General considerations
• maximise attribution
– Ensure no factor other than the intervention differs
between the intervention and control group
– Random allocation, if adequately carried out, will
in the long run ensure comparable groups with
respect to all factors
• minimise all sources of error
– systematic error (bias)
– random error (chance)
• be practical and ethical

46. Health Services Research Unit
Minimise sources of errorMinimise sources of error
Systematic errors (bias)
• “inaccuracy which is different in its size or direction in
one of the groups under study than the others ”
• Minimise bias by ensuring that the methods used are
applied in the same manner to all subjects
irrespective of which group they belong to.
Random errors (chance)
• “Inaccuracy which is similar in the different groups of
subjects being compared”
• Adequate sample size, accurate methods of
measurement
Elwood (1998)

47. Health Services Research Unit
Study designsStudy designs
• Experimental (Randomised controlled trial)
– A new intervention is deliberately introduced and
compared with standard care
• Quasi-experimental (non-randomised, controlled
before and after)
– Researchers do not have full control over the
implementation of the intervention (“opportunistic
research”)
• Observational (Cohort, case-control, cross-sectional)
– describes current practice
– observed differences cannot be attributed solely to
a “treatment” effect

48. Health Services Research Unit
Evaluation of health careEvaluation of health care
interventionsinterventions
• Randomised controlled trials are considered as the
“gold standard”
• However, some debate over the advantages and
disadvantages of different research designs for
assessing the effectiveness of healthcare
interventions
• Polarised views
– “observational methods provide no useful means of
assessing the value of a therapy” (Doll, 1993)
– RCTs may be unnecessary, inappropriate,
impossible or inadequate (Black, 1996)
• Approaches should be seen as complementary and
not as alternatives (Black, 1996)
– Interpretation of RCTs in terms of
generalisability

49. Health Services Research Unit
Useful/interesting linksUseful/interesting links
• www.jameslindlibrary.org (History)
• www.consort-statement.org (CONSORT)
• www-users.york.ac.uk/~mb55/pubs/pbstnote.htm (All
the stats notes from BMJ)
• www.ctu.mrc.ac.uk (MRC CTU)
• www.rcrt.ox.ac.uk (under construction)
• Doll R. Clinical Trials the 1948 watershe BMJ 1998;317:1217-
1220
• The unpredictability paradox: review of empirical
comparisons of randomised and non-randomised clinical
trials Regina Kunz and Andrew D Oxman BMJ 1998 317:
1185-1190

50. Health Services Research Unit
ReferencesReferences
• Black. Why we need observational studies to
evaluate the effectiveness of health care. BMJ 1996:
312;1215-8
• Crombie. Research in Health Care. 1996
• Doll. Doing more good than harm: the evaluation of
health care interventions. Ann NY Acad Sci
1993:703;310-13
• Elwood M. Critical appraisal of epidemiological
studies and clinical trials. 1998 OUP; Oxford.
• Greenhalgh T. How to read a paper. 2001 BMJ;
London
• Schulz and Grimes. Lancet Epidemiology series.
2002