This document discusses various quantitative measures used in therapy articles, including p-value, confidence interval, event rates, relative risk, relative risk reduction, absolute risk reduction, and number needed to treat or harm. It provides examples and formulas for calculating each measure. The document aims to help readers understand how to interpret results reported in therapy studies.
1. Critique of Therapy
Article – Results
Dr. Majdi N. Al-Jasim
SBFM, ABFM
Consultant Family Physician
PCFCM - AlAhsa
2. OBJECTIVES
To Discuss the Quantitative Measures Used in Therapy Articles:
▪ P-value
▪ Confidence Interval
▪ Event Rates
▪ Relative Risk
▪ Relative Risk Reduction
▪ Absolute Risk Reduction
▪ Number Needed to Treat or to Harm
Dr. Majdi Al-Jasim; SBFM, ABFM
3. Validity
Results
Apply it
Is the study done in a correct way?
(Methodology Section)
Are results significant?
(Results Section)
What are benefits and risks?
Is this study generalizable?
Randomized Controlled Trial Critique
Dr. Majdi Al-Jasim; SBFM, ABFM
6. P-value
▪ Used to declare Statistical Significant results.
▪ It emphasizes that how the results occur by chance.
▪ The commonly accepted cut point for calling a result
“statistically significant” is p<0.05
Example:
In a multiple choice question with 5 choices, there is a chance by
20% that you can answer the question correctly even if you did
not study well.!!!
So we can say the p-value here is 0.2
Dr. Majdi Al-Jasim; SBFM, ABFM
7. ▪ A range of values that is almost sure to contain the true
population parameter.
▪ 95% CI means: if we repeat the study 100 times, the results in
95 studies will be within the CI range.
Example:
If 95% CI (5 – 13), then we are sure by 95% that if we repeat the
study again and again, the result will be between 5 and 13.
Confidence Interval (CI)
Dr. Majdi Al-Jasim; SBFM, ABFM
8. ▪ A narrow range CI, means more precision; and it means a
large sample size most likely used in that study.
Example:
Confidence Interval (CI)
95% CI (3 – 25)
Wide range CI Small sample
95% CI (0.7 – 1.2)
Narrow range CI Large sample
Dr. Majdi Al-Jasim; SBFM, ABFM
9. ▪ In studies that use any ratio in their outcome (like Relative
Risk, Odds Ratio), if CI range crosses the value “1”, then the
result is not statistically significant.
▪ In studies that use difference in their outcome (like weighted
mean difference), if CI range crosses the value “0”, then the
result is not statistically significant.
Confidence Interval (CI)
Dr. Majdi Al-Jasim; SBFM, ABFM
10. Confidence Interval (CI)
95% CI (0.2 – 0.7)
“Significant”
95% CI (0.9 – 6.7)
“Not significant”
95% CI (6.8 – 9.7)
“Significant”
Dr. Majdi Al-Jasim; SBFM, ABFM
11. Which of the following 95% CI represent significant results and
large sample size, assuming the result is a ratio:
95% CI (12 – 105)
Significant but small sample size
95% CI (0.8 – 1.2)
Not significant but large sample size
95% CI (0.1 – 0.4)
Significant with large sample size
Confidence Interval (CI)
Dr. Majdi Al-Jasim; SBFM, ABFM
14. Event Rate
The rate of having new outcomes (i.e. incidence) in the group
who took the new treatment (Experiment Event Rate) or the
other group who took old treatment or placebo (Control Event
Rate).
Formula:
Experiment Event Rate (EER) = Events / Group Total
Control Event Rate (CER) = Events / Group Total
Dr. Majdi Al-Jasim; SBFM, ABFM
16. Relative Risk (RR)
The risk of having the outcome (harm or benefit) among
experiment group to those in control group.
➢ Some authors call it Risk Ratio.
➢ If it is over a period of time, it is called Hazard Ratio (HR).
Formula:
RR = Experiment Event Rate / Control Event Rate
RR = EER / CER
Dr. Majdi Al-Jasim; SBFM, ABFM
18. Interpretation:
▪ Harmful outcome:
➢ > 1 the experiment increases the harm.
➢ < 1 the experiment decreases the harm.
➢ = 1 no difference between experiment or control group.
▪ Beneficial outcome:
➢ > 1 the experiment increases the benefit .
➢ < 1 the experiment decreases the benefit.
➢ = 1 no difference between experiment or control group.
Relative Risk (RR)
Dr. Majdi Al-Jasim; SBFM, ABFM
19. Kaplan Meier Hazard Function Plot
Relative Risk (RR)
Death from Cardiovascular
causes in Empagliflozin
group is 5 within 4 years.
Death from Cardiovascular
causes in placebo group is 9
within 4 years; p<0.001
Conclusion:
Empagliflozin is better than
placebo in decreasing the
death from cardiovascular
causes within 4 years.
Always…
In Hazard
Function Plot,
the lower curve
is better than
the upper curve
if the outcome is
harmful and vice
versa for
beneficial
outcome.
Dr. Majdi Al-Jasim; SBFM, ABFM
20. In an article, the risk of having stroke in group taking aspirin is 0.6 times
compared to the group who takes placebo; 95% CI (0.4 - 0.7).
What do you understand from the result?
Relative Risk (RR)
Dr. Majdi Al-Jasim; SBFM, ABFM
21. In an article, the risk of having stroke in group taking aspirin is 0.6 times
compared to the group who takes placebo; 95% CI (0.4 - 0.7).
Analysis:
RR = 0.6
We noticed that RR < 1
The outcome is stroke (harmful outcome).
So, the aspirin will decrease the stroke events.
95% CI (0.4 - 0.7) is not crossing the value “1”, so the result is statistically
significant. Also it is narrow which means large sample size used in this
study.
But for how much?
Relative Risk (RR)
Dr. Majdi Al-Jasim; SBFM, ABFM
22. In an article, the risk of having stroke in group taking aspirin is 0.6 times
compared to the group who takes placebo; 95% CI (0.4 - 0.7).
Here RR is 0.6 which means the risk of stroke in aspirin group is 60%
of that in control group.
In other ward, if control group has risk to develop stroke by 1000
cases in million then the aspirin group will be 60% of that (i.e. 600
cases in million). So the actual decrease is 40% (i.e. 400 cases). That
is why they came up with another measuring parameter called
Relative Risk Reduction (RRR).
Relative Risk (RR)
Dr. Majdi Al-Jasim; SBFM, ABFM
23. Relative Risk (RR)
▪ Relative Risk (RR) just tell if there is
increase or decrease in the outcome
in experiment group; but it doesn’t
tell for how much.
▪ The actual decrease or increase is
calculated as the rest of RR and so
here is where we need to use
Relative Risk Reduction (RRR).
Dr. Majdi Al-Jasim; SBFM, ABFM
24. Relative Risk Reduction (RRR)
Used to see how much the experiment treatment is
relatively reducing the chance of having outcome in
treated patient.
➢If it measures a beneficial outcome, it is called in some
articles Relative Benefit Increment (RBI).
Formula:
RRR = 1 - RR
Dr. Majdi Al-Jasim; SBFM, ABFM
25. Interpretation:
Using experiment treatment will relatively reduce the risk of having
the outcome by (%) compared to control treatment.
Example:
If RRR = 70% in comparing ACEI vs placebo in decreasing IHD. This
means in a person who is treated with ACEI, his chance of having IHD
will be relatively reduced by 70%.
Relative Risk Reduction (RRR)
Dr. Majdi Al-Jasim; SBFM, ABFM
26. In an article, the rate of developing IHD in the group using aspirin was
40%, while it was 54% in the control group.
By how much aspirin will relatively reduce the events
of IHD?
Relative Risk Reduction (RRR)
Dr. Majdi Al-Jasim; SBFM, ABFM
27. In an article, the rate of developing IHD in the group using aspirin was
40%, while it was 54% in the control group.
Relative Risk Reduction (RRR)
Answer:
EER = 40%
CER = 54%
RR = EER / CER = 40/54 = 0.74
RRR = 1 – RR = 1 – 0.74 = 0.26
So RRR = 0.26 = 26%
So using aspirin will relatively reduce the IHD events by 26%.
Dr. Majdi Al-Jasim; SBFM, ABFM
29. ▪ The problem with RRR is that it is bonded to RR.
▪ You can assume wrongly that the treatment is effective
in decreasing the risk of a disease whether it decreases
the risk from 4 per 1000,000 to 2 per 1000,000 or from
4 per 10 to 2 per 10 (both will have RRR = 50%).
▪ Here is where they came up with Absolute Risk
Reduction (ARR).
Relative Risk Reduction (RRR)
Dr. Majdi Al-Jasim; SBFM, ABFM
30. Absolute Risk Reduction (ARR)
Used to see the magnitude of benefit between experiment
treatment and control treatment.
➢ Some authors call it Attributable Risk or Risk Difference.
➢ If it measures a beneficial outcome, it is called in some articles
Absolute Benefit Increment (ABI).
Formula:
ARR = CER – EER
Dr. Majdi Al-Jasim; SBFM, ABFM
31. Interpretation:
if 100 patients were treated with experiment treatment, (x)
cases of outcome can be prevented; or using the experiment
treatment will absolutely reduce the outcome by (%).
Example:
If ARR = 15% in comparing ACEI vs placebo in decreasing IHD. This
means if 100 patients were treated with ACEI, 15 cases of IHD can
be prevented compared to placebo.
Absolute Risk Reduction (ARR)
Dr. Majdi Al-Jasim; SBFM, ABFM
32. In an article, the rate of developing IHD in the group using aspirin was
40%, while it was 54% in the control group.
By how much aspirin will absolutely reduce the
events of IHD?
Absolute Risk Reduction (ARR)
Dr. Majdi Al-Jasim; SBFM, ABFM
33. In an article, the rate of developing IHD in the group using aspirin was
40%, while it was 54% in the control group.
Absolute Risk Reduction (ARR)
Answer:
EER = 40%
CER = 54%
ARR = CER – EER = 54 – 40 = 14%
So using aspiring will absolutely reduce the IHD by 14%.
That is, if 100 patients took aspirin, 14 cases of IHD can be
prevented.
Have you noticed
that how ARR is
smaller than RRR?
Dr. Majdi Al-Jasim; SBFM, ABFM
37. ▪ ARR some times becomes not easy to explain for some
readers, especially if we use both ARR and ABI.
▪ Here where they came up with easy way to interpret
the results in weighting the risk versus benefit; that is
Number Needed to Treat (NNT) and Number Needed to
Harm (NNH).
Absolute Risk Reduction (ARR)
Dr. Majdi Al-Jasim; SBFM, ABFM
38. Widely used to weight risk versus benefit from used
treatment.
Number Needed to Treat / Harm
Benefits
Risks
Dr. Majdi Al-Jasim; SBFM, ABFM
39. ▪ Preventing a harmful outcome:
➢ This is Number Needed to Treat (NNT). It is used to see how
many individuals needed to take the treatment in order to
prevent one bad event.
▪ Causing a harmful outcome:
➢ This is Number needed to harm (NNH). It is used to see how
many individuals needed to take the treatment in order to
develop one bad event.
Formula:
NNT or NNH = 1 / ARR
Number Needed to Treat / Harm
Dr. Majdi Al-Jasim; SBFM, ABFM
40. In an article, the rate of developing IHD in the group using aspirin was
40%, while it was 54% in the control group.
Number Needed to Treat / Harm
How many patients we need to treat with aspirin in
order to prevent one case of IHD?
Dr. Majdi Al-Jasim; SBFM, ABFM
41. Answer:
EER = 40%
CER = 54%
ARR = CER – EER = (54 – 40) = 14%
So ARR = 14% = 0.14
NNT = 1 / ARR
NNT = 1 / 0.14
NNT = 7.14
So we need to treat 7 patients with aspirin in order to prevent
one IHD event.
Number Needed to Treat / Harm
Calculation tip:
If you use ARR in (%), then
calculate NNT as
following:
NNT = 100 / ARR (%)
Dr. Majdi Al-Jasim; SBFM, ABFM
42. In the same article, the rate of developing hemorrhagic stroke in the group
using aspirin was 35%, while it was 19% in the control group.
Number Needed to Treat / Harm
How many patients we need to treat with aspirin in
order to cause one case of hemorrhagic stroke?
Dr. Majdi Al-Jasim; SBFM, ABFM
43. Answer:
EER = 35%
CER = 19%
ARR = EER – CER = (35 – 19) = 16%
So ARR = 16% = 0.16
NNT = 1 / ARR
NNT = 1 / 0.16
NNH = 6.25
So we need to treat 6 patients with aspirin in order to cause one
hemorrhagic stroke event.
Number Needed to Treat / Harm
Calculation tip:
If you use ARR in (%), then
calculate NNH as
following:
NNH = 100 / ARR (%)
Dr. Majdi Al-Jasim; SBFM, ABFM
44. Is it NNT or NNH that we need to use in this question?
Number Needed to Treat / Harm
Dr. Majdi Al-Jasim; SBFM, ABFM
45. Is it NNT or NNH that we need to use in this question?
If the bad events rate in control group are more than that in experiment
group (i.e. CER > EER), then it is NNT and vice versa for good events.
Example:
Number Needed to Treat / Harm
Bad
events
EER = 10.5%
CER = 12.1%
The experiment treatment causes less harmDr. Majdi Al-Jasim; SBFM, ABFM
46. Is it NNT or NNH that we need to use in this question?
If the bad events rate in experiment group are more than that in control
group (i.e. EER > CER), then it is NNH and vice versa for good events.
Example:
Number Needed to Treat / Harm
Bad
events
EER = 38.3%
CER = 37.1%
The experiment treatment causes more harm
Dr. Majdi Al-Jasim; SBFM, ABFM
47. Summary
Check the significance and how the results happen by
chance
Compare the risk in experiment versus control group
How much the risk is relatively reduced in treated
patient
Absolute reduced risk in 100 treated patients
Dr. Majdi Al-Jasim; SBFM, ABFM
48. The number needed to prevent one harm event
The number needed to cause one harm event
We are certain that the repeated results will be within
this range
Summary
Dr. Majdi Al-Jasim; SBFM, ABFM
50. For more EBM interactive lectures, I highly recommend
you visiting Prof Terry Shaneyfelt YouTube Channel:
https://www.youtube.com/user/UABEBMcourse/videos
Yours;
Dr. Majdi Al-Jasim