2. Who should be analyzed?
Measuring Outcomes
Effect measures
Clinical and statistical significance
Reporting the trial
Lecture Outline
3. Recap: Types Of Clinical Trials
By Outcome Of Interest (Efficacy Vs. Effectiveness)
Test efficacy
Highly selected
participants
Highly Controlled
conditions
Test effectiveness in
every day practice
Unselected Participants
Flexible Conditions
Give informed decisions
about practice
Explanatory RCTs Pragmatic RCTs
Efficacy:
Investigates the benefits of an
intervention under ideal
and highly controlled
conditions
Effectiveness:
Examine the outcomes of
interventions under
circumstances that more closely
approximate the real world
4. Recap: Types Of Clinical Trials
By Outcome Of Interest (Efficacy Vs. Effectiveness)
Test efficacy
Highly selected
participants
Highly Controlled
conditions
Test effectiveness in
every day practice
Unselected
Participants
Flexible Conditions
Explanatory RCTs Pragmatic RCTs
Efficacy:
Investigates the benefits of an
intervention under ideal and
highly controlled conditions
Effectiveness:
Examine the outcomes of
interventions under circumstances
that more closely approximate the
real world
5.
6. Efficacy (per protocol) :
The extent to which a specific intervention, procedure,
regimen, or service produces a beneficial result under
ideal conditions;
This means the patient actually took all doses of
treatment,
And all elements of the protocol followed (ie full
follow-up)
Optimal Estimate: Answers the patient’s question
“Doctor, if I take this drug, will I get better?”
Efficacy vs Effectiveness
7. Efficacy vs Effectiveness
Effectiveness (intention to treat)
The effect of a specific intervention, procedure, regimen, or
service, when deployed in the field in routine circumstances.
This accounts for non-compliance, dropouts and side effects.
All patients randomized (allocated to treatment) are analysed,
whether or not they completed the prescribed regimen, and
follow-up.
Conservative estimate: Answers the public health question
“What is the overall effect of this treatment given to a
population?”
9. Dealing with losses to follow-up and
non-compliance
So, assume we have randomised the patients to the two
treatments and given them the drugs
We see them at intervals to monitor progress and at the
pre-defined end-point to determine the outcome
Losses to follow-up:
After randomization, participants may overtly or
covertly not take the assigned treatment. They “do not
comply” with their allocation
Not everyone continues in the trial
10. Reasons for withdrawal of Subjects
TREATMENT COMPLIANCE = the degree to which participants
and personnel adhere to the treatment protocol
Noncompliance (adverse effects of intervention, loss of interest,
changes in underlying conditions, substance usage)
Drop-outs are the participants who do not
adhere to the experimental regimen during
follow-up
Drop-ins are the participants who do not
adhere to the control regimen during
follow-up
11. Use of Intention-to-
Treat analysis in an
unplanned crossover.
Need to deal with:
Losses to follow-up
and
Non-compliance
appropriately in the
analysis
12. ❑ Also called Analysis by assigned treatment
❑ “once randomized always analysed”
❑ Analysis according to how they were randomized treatment
assignment and NOT to which treatment they ended up getting
(e.g. if they swapped or dropped out)
❑ „Ignore noncompliance, protocol deviations, withdrawal, and
anything that happens after randomization
❑ The Recommended method of analysis
❑ Measures effectiveness (will it work? To determine the effect in
routine practice)
❑ Less biased results than “as treated” because you maintain
randomization.
Primary analysis (Intention-to-treat analysis):
13. ❑ Also called Analysis by assigned treatment
❑ “once randomized always analysed”
❑ Analysis according to how they were randomized treatment
assignment and NOT to which treatment they ended up getting
(e.g. if they swapped or dropped out)
❑ „Ignore noncompliance, protocol deviations, withdrawal, and
anything that happens after randomization
❑ .
Primary analysis (Intention-to-treat analysis):
14. ❑ The Recommended method of analysis
❑ Measures effectiveness (will it work? To determine the effect in
routine practice)
❑ Less biased results than “as treated” because you maintain
randomization.
Primary analysis (Intention-to-treat analysis):
15. 15
Reality of what happens
Randomise
A B
Take Take
Don’t take Don’t take
16. 16
Pragmatic (practical) trial – ‘analysis by
intention to treat’
• In routine clinical practice is A better
than B ?
Randomise
A B
Take Take
Don’t take Don’t take
Compare by ‘intention to treat’ – regardless of
whether they actually took the treatment!
17. e.g. Intention to Treat Analysis
5000 Patients Screened
1000 Randomized
500 Placebo
500 New ttt
Outcome
Outcome
23 withdraw
consent
14 lost to
followup
22 stop taking
medicine
10withdraw
consent
4 lost to followup
13 stop taking
medicine
?
Number improved /
500
Number improved /
500
18. ITT analysis
❑ Golden standard for data analysis
❑ Usually provides a conservative estimate of treatment effect
❑ Pragmatic analysis
ADVANTAGES LIMITATIONS
Preserving randomization,
minimizing bias
Does not determine maximum
efficacy of treatment
Depicting real-life situations Large loss to follow-up leads to
inconclusive results
Uses information from all subjects at
any given time
Might not show the potential benefit
or show smaller benefit compared to
PP
Gives practical information on
treatment administration
19. 2ry analysis Per Protocol Analysis (PPA)
❑ All participants are analyzed according to the
treatment they actually received, regardless of what
treatment they were originally allocated. Excluding
non-adherenced persons
❑ Effect of random allocation is compromised, making the
interpretation of the results difficult
❑ Measures efficacy (can it work? To determine the maximum effect under
ideal conditions)
Treatment received / completers analysis
20. 20
Explanatory trials – ‘as treated’ analysis
• Is the physiological action of drug A
better than drug B ?
Randomise
A B
Take Take
Don’t take Don’t take
Ignore those who don’t take the drug
21. Per protocol (PP) analysis
❑ Main principle
❑ Efficacy analysis, explanatory analysis, analysis by
treatment administered
ADVANTAGES LIMITATIONS
Maximal efficiency of
treatment
Non-adherence related to
prognosis
Relation to adverse effects
Undermining
randomization
22. Efficacy (per protocol) :
The extent to which a specific intervention, procedure,
regimen, or service produces a beneficial result under
ideal conditions;
This means the patient actually took all doses of
treatment,
And all elements of the protocol followed (ie full
follow-up)
Optimal Estimate: Answers the patient’s question
“Doctor, if I take this drug, will I get better?”
Efficacy vs Effectiveness
23. Efficacy vs Effectiveness
Effectiveness (intention to treat)
The effect of a specific intervention, procedure, regimen, or
service, when deployed in the field in routine circumstances.
This accounts for non-compliance, dropouts and side effects.
All patients randomized (allocated to treatment) are analysed,
whether or not they completed the prescribed regimen, and
follow-up.
Conservative estimate: Answers the public health question
“What is the overall effect of this treatment given to a
population?”
24. ITT vs PPA
Intention to Treatment Per Protocol Analysis
• Trials should be analyzed in
the groups to which they
were randomized, regardless
of treatment received
• Participants might be
deviated from the protocol
(sometimes)
• Randomization - it is the best
guarantee that the groups of
patients being compared have
similar characteristics
• Analysis of treatment groups
that includes only those
patients who completed the
treatment originally allocated
• Might be useful for analyzing
the adverse effects of
treatments
• The results of this analysis
usually provide a lower level of
evidence but better reflect the
biological effects of new
treatment
26. Analysis by treatment assigned Vs Treatment
received
Argument between trialist and clinician
The trialist demands: analysis by treatment assigned be the
primary analysis whereas,
Clinician is primarily interested in analysis of treatment
administered
Conduct both kinds of analysis and present both in primary
results publications
Solution
27. ITT is more realistic,
PP is falsely optimistic
29. Assessing the Results of an RCT
✓ A trial is conducted to compare treatments, e.g., a new treatment
against the current standard or a placebo.
✓ Results Assessed not only statistically but also clinically
✓ Statistical significance focuses on the null value of the effect.
✓ More important is clinical significance defined by the
smallest clinically beneficial and harmful values of the effect
✓ The effect size, is a measure of the improvement in the new
treatment over the standard.
Statistical significance
P value of the used test
Clinical significance
Number Needed To Treat (NNT)
30. Which drug would you take?
Drug A can reduce your MI risk by 1/3
Drug B can reduce your MI risk by 9 %
Drug C every 11 patients who take Drug C, one MI will
be prevented
A B C
31. ❑ Measures of Effect size
A. Relative risk (RR)
B. Absolute risk reduction (ARR)
C. Relative risk reduction (RRR) = Efficacy
D. Number needed to treat (NNT)
Total
outcome
Group
Negative
Positive
a+b
b
a
Intervention
c+d
d
c
Control
a+b+c+d
b+d
a+c
Total
Measures and Detection of Treatment
Effect
❑ Data set up: familiar 2 x 2 table
32. 1- Relative Risk (RR) – RCT’s
It is the relative probability (or risk) of the event in the
treatment group compared to the control group
Experimental Event Rate (EER)
-----------------------------------------------
Control Event Rate (CER )
EER: The percentage of intervention group who
experienced outcome in question. ( a/(a + b))
CER: The percentage of control group who experienced
outcome in question. (c /( c + d))
35. Experimental Event Rate (EER)
= Risk of outcome event in
experimental group
Control Event Rate (CER)
= Risk of outcome event in
control group
Experiment Event Group
Control Event Group
36. Example 1
Randomised Control Trial of a new drug tested on a
population at risk of a heart attack.
▪ 60/100 of those receiving the drug (experimental
group) will have a heart attack.
▪ Risk in experiment group is: 0.6 = 60% risk of
event
▪ 90/100 of those not receiving the drug (control
group) will have a heart attack.
▪ Risk in the control group is: 0.9 = 90% risk of
event
The intervention has lowered the risk of the event happening
37. Relative Risk (RR)
0.6 ÷ 0.9 0.67
Compares the risk of having an event between two
groups
RR = EER /CER =
÷
38. 2- Relative Risk Reduction (RRR)=Efficacy
It is The proportion of the baseline risk that is removed by
therapy
RRR = 1 – RR
Indicates by how much in relative terms the event rate is
decreased.
Also calculated as the ARR divided by the baseline risk
RRR= (CER – EER) / CER
Clinical interpretation (RCT):
“the death rate is ----% lower after NEW treatment
compared to STANDARD treatment”
Null value = 0.
39. Relative Risk Reduction (RRR)
The reduction in the rate of the event in the
treatment group relative to the control group
RRR = 1 – RR =
High
Risk 1 - 0.67
Low
risk 1 - 0.67
= 0.33 = 33%
= 0.33 = 33%
Relative risk reduction (RRR) tells you by how much the treatment
reduced the risk of the event - in the experimental/intervention
group - relative to the risk of the event in the control group - who
did not have the treatment.
40. 3- Absolute Risk Reduction (ARR)
It is the difference in absolute risk (or probability of events) between
the control and treatment groups
Also be called the risk difference (RD) or attributable risk
ARR= CER - EER
Clinical interpretation (RCT):
“the absolute risk of event (e.g. death) is ------% lower with new
treatment compared to standard treatment”
A simple and direct measure of the impact of treatment
Null value = 0.
41. Absolute Risk Reduction (or Risk Difference)
High Risk 0.9 - 0.6 = 0.3 30%
Low Risk 0.03 - 0.02 = 0.01 1%
Compares the risk of having an event between two groups
ARR = CER – EER =
33%
33%
RRR
42. Example
If you didn't take aspirin, your risk of having a heart attack was 2% over 5 years.
If you did take aspirin, your risk of having a heart attack was 1% over 5 years.
Relative Risk Reduction would say that aspirin reduces your chance of heart attack by 50%
Absolute Risk Reduction would say that aspirin reduced your chance of heart attack by 1%
43. 4- The Number Needed To Treat (NNT)
Number of patients who would have to receive an
intervention for 1 to benefit AND prevent an adverse
event
NNT = 1 / ARR (It is the inverse of ARR.)
E.g. How many people with myocardial infarction would
you have to treat with ß-blockers for 2 years to prevent 1
death?
44. Clinical interpretation (RCT):
If, for example, the NNT for a treatment is 10, the practitioner
would have to give the treatment to 10 patients to prevent 1 patient
from having the adverse outcome over the defined period
A very useful clinical measure because it is more interpretable
than the ARR and it conveys the impact of a clinical
intervention
Interpretation
The smaller the NNT, the larger the differences between the
two drugs, i.e. larger numbers mean more patients needed to
treat to see the difference in effect
45. The number of patients who would have to receive the
treatment for 1 of them to experience the adverse effect
;calculated as 100 divided by the absolute risk increase
The Number Needed To harm
46. P Values vs NNT
P VALUE NNT
Indicates Statistical Significance Indicates Clinical Significance
Value gives an indication of how
strong the likelihood that any
difference is NOT due to chance
Helps judge the clinical
significance of a statistically
significant result
Independent of Effect Size NNT is one measure of effect
size
The smaller the p value, the more
convinced that something is going
on that is not just random
It is independent of p value and
does not say anything about the
likelihood of the difference
between treatments being due to
chance alone
*
47. Example 1
Death Survival Total
Treatment
group
18 46 64
Control
group
29 36 65
Total
In a recent study of pain relief by antidepressants drugs
compared with placebo , the findings were presented in the
table below
Calculate RR, ARR, RRR AND NNT
48. 3-Absolute Risk Reduction (ARR):
ARR= CER - EER
=(29/65) – (18/64) =0.446 – 0.281 = 0.165 = 16.5%
2-Relative Risk Reduction (RRR)
RRR= (CER – EER) / CER
=(0.446 – 0.281) / 0.446 =0.165 / 0.446 = 0.37 = 37%
i.e. new treatment decreases the risk of death by 37%
4-Number Needed to Treat (NNT):
NNT = 1/ ARR = 1 / 0.165 = 6.06 =6 patients
You have to treat 6 patients by new treatment to
save one life
1- Relative Risk (RR): RR= EER/ CER
RR=(18/64) / (29/65) = 0.281/0.446 =0.63 =63%
49. Example 2
Death Survival Total
Treatment
group
13 87 100
Control
group
25 75 100
Total 38 162 200
In a recent study of pain relief by antidepressants drugs
compared with placebo , the findings were presented in the
table below
Calculate RR, ARR, RRR AND NNT
50. Example 3
Death Survival Total
Treatment
group
98 847 945
Control
group
152 787 939
Total
A multi-centre, randomised placebo-controlled trial of the
beta blocking drug Timolol, reported the number of deaths in
18 months of follow-up among patients who had recently
suffered a myocardial infarction.
Calculate RR, ARR, RRR AND NNT
52. Which drug would you take?
Drug A can reduce your MI risk by 1/3
Drug B can reduce your MI risk by 9 %
Drug C every 11 patients who take Drug C, one MI will
be prevented
A B C
53. Which drug would you take?
Drug A can reduce your MI risk by 1/3rd
Relative Risk Reduction (RRR = 33%)
Drug B can reduce your MI risk by 9%
Absolute Risk Reduction (ARR = 9 %)
Drug C prevent an MI for every 11 patients who take it
regularly
Number Needed to Treat (NNT = 11)
55. Choosing The Right Test
1) Type of research question
✓ Test difference or
✓ Assess association between variable or
✓ Predict the value of one variable
2) Level of measurement (nominal, ordinal or scale)
3) Distribution of data ( Normally distributed data or not)
4) Number of the variables being studied ( 2 or more)
5) Paired or independent
Dr. Eman M. Mortada
57. No
Yes
Nominal Data
Are your samples paired?
Chi-square (x²) test
Is there any expected value <5 ?
McNemar’s test
Fisher Exact test
Caffeine Ca No Ca
Yes 25 10
No 25 10
Total 50 20
Dr. Eman M. Mortada
58. Scale Data
Comparing 2 Gps
Independent
t-test
Paired t test
Comparing >2 Gps
ANOVA
Assoc. 2 Gps
Pearson
Correlation
Prediction
Regression
Dr. Eman M. Mortada
59. No
Yes
Scale Data
Paired t test. Subject Pre Post
1 10 12
2 50 52
3 20 25
4 8 10
5 115 120
6 75 80
7 45 50
8 170 175
Dr. Eman M. Mortada
Are the samples paired or dependent?
60. How many groups (samples) do you have?
2
>2
Independent t test
Analysis of Variance
(ANOVA).
Samples are independent
Not-
smoker
Ex-
smoker
Smoker
Patient FEV1 FEV1 FEV1
1 3.40 2.28 2.15
2 3.46 3.34 2.85
3 3.50 3.38 3.86
4 3.56 3.50 3.97
5 4.11 4.40 2.95
6 4.41 3.28 2.98
7 4.51 3.28 3.11
8 4.88 3.54 3.25
9 4.96 3.92 3.41
10 4.98 4.44 3.78
Mean 4.18 3.54 3.23
Dr. Eman M. Mortada
61. Ordinal Data
Comparing 2 Gps
Mann-Whitney
(U) test.
Wilcoxon (Pre-
Post).
Comparing >2 Gps
Kruskal Wallis
Assoc. 2 Gps
Spearman’s ρ
Dr. Eman M. Mortada
62. No
Yes
Ordinal Data
Are your samples paired or dependent?
Wilcoxon (signed rank test)
=PAIRED T TEST
Dr. Eman M. Mortada
63. Ordinal Data
2
>2
How many groups (samples) do you have ?
Kruskal-Wallis’ Test
Mann-Whitney test
=independent t test
=ANOVA
Dr. Eman M. Mortada
65. Dissemination of Results
Research is not complete without dissemination.
Dissemination at local, national, international
level.
Publications in journals, online, conference
presentations, dissemination workshops, website.
Information must reach those who most need it.
66. Reporting guidelines
Guidelines Type of study
CONSORT on reporting of randomized controlled trials
STROBE on reporting of observational studies in
epidemiology
PRISMA on reporting of systematic reviews
MOOSE on reporting of meta-analyses of
observational studies
67. (CONsolidated Standards Of Reporting Trials
is the standard guide for clinical trials reports in medical
publications.
Consists of 25 item checklist and a flow diagram
The checklist items focus on reporting how the trial was designed,
analyzed, and interpreted;
the flow diagram displays the progress of all participants
through the trial.
CONSORT……. Stands for
