Powerpoint presentation on the basic aspects of evidence-based medicine by Dr Max Mongelli, Nepean Hospital, Sydney NSW

2. What is EBM?
“The conscientious, explicit and judicious
use of current best evidence in
making decisions about the care of
individual patients”
Prof. David L. Sackett, 1997

3. Why EBM?

4. Primum non nocere
“First do no harm”
Hippocrates, Epidemics

5. Sources of Evidence in Medicine
• Traditional Teaching
• Textbooks
• Basic sciences
• Observational studies
• Computer simulation
• Decision Analysis
• Case-Control Studies
• Randomised Controlled Trials (RCT)
• Meta-analyses

6. RCOG Classification of Evidence Levels
1++ High quality meta-analyses of RCT's
1+ Meta-a. Or RCT's at low risk of bias
1- Meta-a. Or RCT's at high risk of bias
2++ High quality meta-analyses of CC's
2+ Well-conducted cc or cohort studies
2- Case-control or cohort studies with ? bias
3 Case reports
4 Expert opinion

7. Archie Cochrane (1909-88)
“Effectiveness and
Efficiency: Random
Reflections of Health
Services “, 1971

8. Randomized Controlled Trials
“Gold standard” in evaluating new
therapies or surgical techniques
May also be applied to new diagnostic
tests

9. Randomized Controlled Trials
Objectives of RCT:
Minimize bias by randomisation
Achieve statistical power through
adequate sample size
“Blinding “ - single or double
Analysis by intention to treat

10. Randomized Controlled Trials
Randomisation
Several techniques available
Computer software linked to central
monitoring station
“Block “ randomisation
Sealed envelope method

12. RCT’s and Observational Studies
• Two studies published in the NEJM in 2000 suggested that
RCTs and observational studies overall produced similar
results
• JAMA 2001: “discrepancies beyond chance do occur and
differences in estimated magnitude of treatment effect are
very common”
• RCTs may be unnecessary for treatments that have
dramatic and rapid effects relative to the expected

13. RCT’s and Industry Funding
•RCT’s funded by industry are significantly more likely to
report positive results
•Possibly due to publication bias
•RCTs may be unnecessary for treatments that have
dramatic and rapid effects relative to the expected

14. RCT’s and Statistical Error
• Type I error – “false positive”
• Type II error – “false negative”
• Sample size calculations often inaccurate

15. Diagnostic Tests

16. 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative

17. 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative
Sensitivity = TP =

18. 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative
Sensitivity = TP = a/(a+c)

19. 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative
False Positive Rate = FP =

20. 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative
FP = b/(d+b)

21. 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative
Specificity = 1 - FP =

22. 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative
Specificity = 1 - FP = d/(d+b)

23. 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative
Positive predictive value (PPV) =

24. 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative
PPV = a/(a+b)

25. 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative
Negative predictive value (NPV) =

26. 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative
NPV = d/(c+d)

27. PPV and Prevalence
Steep drop in positive predictive
value as disease prevalence
decreases

28. The Likelihood Ratio
Single value to indicate the
clinical utility of a test
Independent of prevalence
LR = Sensitivity/(1- Spec.)
LR >8 : tests usually clinically
useful

29. The Likelihood Ratio
LR is an odds modifier:
Posterior odds =
prior odds x LR

30. Odds and Probability
Inter-convertible:
Odds = p/(1-p)

31. Can tests be combined ?
Rare conditions: high rates of
false positives
Lead to excessive unnecessary
intervention
Can be reduced by combining
tests e.g. intrapartum
monitoring

32. Impact of new diagnostic
test on clinical outcomes:
RCT
Cohort study
Case-control study
Before and after study

33. SYSTEMATIC REVIEWS

34. "It is surely a great criticism of our
profession that we have not organised
a critical summary, by specialty or
subspecialty, adapted periodically, of
all relevant randomized controlled
trials."
Archie Cochrane, 1972

35. Role of systematic reviews
Before commencing a new project: to determine
whether further studies are really indicated: ‘state-
of-the-art’ literature review.
Gain in statistical power for average estimates.
'Cumulative' meta-analysis can determine when
further studies are no longer indicated.
Design of subsequent studies.
Setting policy for treatment and health care –
making the best use of the data available.

36. Can Studies be Combined?
Identification of optimal inclusion criteria can be
difficult.
The most critical step is choosing the appropriate
research question.
A fairly general question is more preferable to a very
specific one.
Tukey : "...far better an approximate answer to the
right question, than an exact answer to the wrong
question.."

37. Publication Bias
Entire research studies may fail to reach publication
because of the nature of the results.
Identification of unpublished trials can be very
difficult - in one study it accounted for 22% of the
papers included in the meta-analysis.
Failure to publish rests with the investigators rather
than editors.

38. PREDICTIVE ABILITY OF META-ANALYSES
Villar et al, Lancet 1995
Comparison of the meta-analyses of smaller studies
with the corresponding result of the largest study.
30 meta-analyses including a total of 185 randomised
controlled studies (RCT) obtained from the Cochrane
pregnancy and childbirth database. The meta-
analyses were only included if they had at least one
trial with a total sample size of over 1000.
Calculations differ from the Cochrane database in
that the largest trial was excluded, this being used as
the 'gold standard' for outcome

40. The Cochrane Collaboration
• Established in 1993 by Sir Iain Chalmers
• International: 100 countries
• Independent
• Not-for-profit
• Over 27000 contributors

41. RANZCOG and EBM
“RANZCOG endorses the principles of Evidence-
based Medicine and recognizes the NHMRC levels of
evidence and grades of recommendations”
College Statement C-Gen 15, Nov. 2009

42. …a scientific idea can never be proven true,
because no matter how many observations seem to
agree with it, it may still be wrong. On the other
hand, a single contrary experiment can prove a
theory forever false
Sir Karl Popper