This document discusses evidence-based medicine and different types of medical evidence and study designs. It begins by defining evidence-based medicine as applying the best available evidence from clinical research to medical decision making. It then discusses different types of medical studies and evidence, including randomized controlled trials, observational studies, systematic reviews, and qualitative research. It emphasizes that not all medical evidence is equal and that higher quality evidence comes from studies with rigorous designs like randomized controlled trials that minimize bias. The document stresses the importance of critically appraising medical evidence to assess its validity, clinical importance, and applicability to a specific patient.

1. Evidence Based Medicine

2. Approaching this module
• Understanding EBM
• What is it?
• Why do we need it?
• What are the benefits?
• Who benefits?

3. What form of Medicine are you going to study?
• Allopathy?
• Homeopathy?
• Ayurveda?
• Western Medicine?
• Modern Medicine?

4. What is NOT Evidence Based Medicine
• CAM
• Complementary and Alternative Medicine
• CAM is largely based on false science, pseudoscience or unverified/untested
experiences and traditions
• “Assertions made without evidence can be dismissed without evidence”
• Understanding what good science isn’t is a good way to learn what good
science is. Learn to spot what’s wrong in flawed science.

5. Our Ancestors didn’t have “Science”
Only Two ways of Learning
• Personal Experience
• What others told them
EVOLUTION prepared them to
• Pay attention to stories
• See patterns
• Make fast decisions
Importance of pattern recognition
IS that a lion or just some
shadow?
If you wrongly think it’s a lion
you run needlessly
If you wrongly think it’s some
shadow, you die.
What seems like a more
rational choice?
EVOLUTION and Pattern recognition:
GOOD: Because it helps us make sense of our environment; act in time,
emotions motivate us to act
BAD: Because we also see patterns that aren’t there; jump to conclusions,
emotions interfere with judgement

6. Allopathy
Greek; állos, meaning "other”, páthos, meaning "suffering”
Term originally coined by Samuel Hahnemann (the father of
Homeopathy) as a derogatory term for traditional European
(Heroic) medicine of the time.
Heroic Medicine: treating by opposite; based on balance of the
four "humours" (blood, phlegm, yellow bile, and black bile) where
disease was caused by an excess of one humour, and would thus
be treated with its opposite
“Remember that the term "Allopath" is a false
nickname not chosen by regular physicians at all, but
cunningly coined, and put in wicked use against us, in
his venomous crusade against Regular Medicine by
its enemy, Hahnemann, and ever since applied to us
by our enemies with all the insinuations and derisive
use the term afford.”
Cathel DW and Cathel W. Book on the Physician Himself, Philadelphia: Davis, 1902, pp.300-301

7. Homeopathy
• A system of alternative medicine created in 1796 by
Samuel Hahnemann, based on his doctrine of like cures like
(similia similibus curentur), a claim that a substance that
causes the symptoms of a disease in healthy people would
cure similar symptoms in sick people.
• Homeopathy is a pseudoscience – a belief that is
incorrectly presented as scientific.
• Homeopathic preparations are not effective for treating
any condition
Concept of Dilution in Homeopathy
• "centesimal" or "C scale", diluting a substance
by a factor of 100 at each stage.
• In homeopathy, a solution that is more dilute is
described as having a higher "potency"
A popular homeopathic treatment for the flu is a 200C dilution of
duck liver, marketed under the name “Oscillococcinum”. As there
are only about 1080 atoms in the entire observable universe, a
dilution of one molecule in the observable universe would be about
40C. Oscillococcinum would thus require 10320 more universes to
simply have one molecule in the final substance

8. “Bloodletting” in medicine
Withdrawal of blood from a patient to prevent or cure illness and disease.
Therapeutic phlebotomy.
• Everybody knew it worked…
• Medical Textbooks explained how it worked…
• Everybody “knew” it saved lives as they saw it with their own eyes…
In 1833 alone, France imported 42 million leeches for medical use.
However, controlled trials only showed that bloodletting actually killed
more people than non-treatment
A chart showing the parts of the body to be bled for different diseases, c. 1310–1320

9. Evidence Based Medicine
• Evidence-based medicine (EBM) aims to apply the best available evidence gained from the
scientific method to medical decision making. It seeks to assess the strength of evidence
of the risks and benefits of treatments (including lack of treatment) and diagnostic tests.

10. Evidence Based Medicine
vs
Science Based Medicine
• EBM; a vital and positive influence on the practice of medicine, but it has limitations and
problems in practice: it often overemphasizes the value of evidence from clinical trials
alone, with some unintended consequences, such as taxpayer money spent on “more
research” of questionable value.
• “the effectiveness of parachutes has not been subjected to rigorous evaluation by using
randomized controlled trials.” BMJ. 2003 Dec;327(7429):1459–61
• Science-Based Medicine is dedicated to evaluating medical treatments and products of
interest to the public in a scientific light, and promoting the highest standards and
traditions of science in health care.

11. Paradigms: They are changing
Old paradigm  New paradigm
Unsystematic clinical experience  Systematic clinical experience
Pathophysiology  Pathophysiology necessary but not sufficient
Content expertise & authoritarianism Rules of evidence

12. EBM
EBM = Good Clinical Practice
Knowledge from best external evidence based on clinical research
+
Judgement from experience
+
Understanding of patient's beliefs & preferences.
Integrating best external evidence with individual clinical expertise
BMJ 1996;312:71-72 (13 January)

13. Clinical Expertise
EBM
Conscientious, explicit and judicious
use of current best evidence in
making decisions about the care of
an individual patient
Sackett. BMJ 1996;312:311-2

14. Are there any pre-requisites for practicing EBM?

15. 5 Steps to EBM
ASK
Ask the RIGHT Clinical
Question
Formulate a searchable
question
ACQUIRE
Collect the most relevant
publications:
Efficient Literature
Searching
Select the appropriate &
relevant studies
APPRAISE
Critically appraise and
synthesize the evidence
APPLY
Integrate best evidence
with personal clinical
expertise, patient
preferences and values:
Applying the result to
your clinical practice and
patient.
ASSESS
Evaluate the practice
decision or change:
Evaluating the outcomes
of the applied evidence
in your practice or
patient.
We Will Review Each One by One

16. ASSESS… At Every STEP

17. Why is asking a Good Question
So Important
ASK
•A good answerable question will help us focus on relevant evidence (to a patient’s clinical
needs, or your own knowledge needs).
•They can suggest high yield search strategies
•They can suggest the forms that useful answers might take
(i.e. what is best research design to answer our question)
Asking good questions is a skill to be learned.

18. Asking the Right Questions
ASK
Clinical Process and Knowledge
requirements
Etiology
Knowledge about causation
Patient Presentation
Knowledge about diagnosis
Diagnosis
Knowledge about prognosis
Clinical Outcome
Knowledge about therapeutic
effectiveness
Testing
History
Examination
Investigations
Therapy
Changes Prognosis
Etiognostic Research
Diagnostic Research
Prognostic Research
Therapy Research

19. Remember…
ASK
• There is no right or wrong way to turn a scenario/knowledge need into a
question.
• Just make sure it is concise, clinical and uses appropriate language (avoid
colloquialism, in favor of technical language).
• Ask one question at a time.

20. There are Tricks…
ASK
PICO(D)
Population-Intervention-Comparison-------------Outcome----------------------------------(Design)
FOREGROUND Question
vs
BACKGROUND Question
• Information found in primary literature, evidence-based extraction service,
guidelines, or systematic reviews
• Questions about issues of care, i.e diagnostic tests or therapies, needed for
clinical decision-making
• Information can be found in textbooks and review articles
• General questions about conditions, illnesses, syndromes and patterns of
disease, pathophysiology
standard therapy, “gold standard” efficacy of therapy, mortality, specificity of diagnostic test
Be specific

21. FRAMING THE QUESTION BY “PICO”
• ELEMENT PROMPTS THE QUESTION:
• Patient How would I describe a group of patients similar to mine?
• Intervention What main action am I considering?
• Comparison What is/are the other options?
• Outcome What do I (or the patient) want to happen (or not happen)?
Example:
• P: In kids under age 12 with poorly controlled asthma on metered dose inhaled steroids…
• I: would the addition of salmetrol to the current therapy
• C: compared to increasing the dose of current steroid
• O: lead to better control of symptoms without increasing side effects?
ASK

22. An Example of “PICO”
In the following questions, identify the missing component:
Is oral rehydration in the emergency room more cost-effective than IV
rehydration?
• Patient/Population – Not identified. Examples: infants, infants with
vomiting.
• Intervention – Oral rehydration.
• Comparison – IV rehydration.
• Outcome – Cost-effectiveness.
ASK

23. Another Example of “PICO”
In the following questions, identify the missing component:
Will “Atrovent” help prevent hospitalization of my 2 year old patient with an
acute asthma exacerbation?
• Patient/Population – Child w/ acute asthma.
• Intervention – Atrovent.
• Comparison – Not identified. Examples: standard therapy, albuterol alone.
• Outcome – Prevent hospitalization.
ASK

24. An-Another Example of “PICO” 
In the following questions, identify the missing component:
Is 10 days of antibiotic therapy better than 5 days of antibiotic therapy for
treating an infant with acute otitis media?
• Patient/Population – Infant with AOM.
• Intervention – 10 days of antibiotic therapy
• Comparison – 5 days of antibiotic therapy.
• Outcome – Not identified. Examples: Resolution of symptoms, recurrence risk,
persistent effusion.
ASK

25. Quick Quiz: Background Q or Foreground Q?
• What causes gastroenteritis?
• Is oral rehydration as effective as IV rehydration?
• How can I tell if my patient is dehydrated?
Background
Foreground
Background
• What are the symptoms of otitis media? Background
• Can I effectively treat otitis media with a shorter
course of antibiotics?
Foreground
• Can this febrile infant be safely treated as an
outpatient?
Foreground

26. Do Not Ignore A Layperson’s Question
THE PATIENT’S QUESTIONS Must be considered and addressed!
Often QUALITATIVE (not based on measureable outcomes)
Feelings, ideas, experiences, preferences, concerns, fears, beliefs, ethnicity
Usually based on LIMITED BACKGROUND
• Perception of problem
• Self-diagnosis
• Treatment wanted or needed
• Alternatives (read, heard, considered, tried)
• What is the patient hoping to avoid?
• What benefits does the patient want or need most?
• Etc.
ASK

27. Expert’s Question: Quantitative
Layperson’s Question: Qualitative
ASK
QUANTITATIVE: “Solid Evidence”
• Measurable answer or response
• Necessary for scientific study
• Necessary for the practice of EBM
QUALITATIVE: “Quality of Life”
• “Fuzzy” data - Impact on daily life, work, family, etc.
• May be very important and influential to decisions – especially for the patient
• Creates added challenge or twist to practice of EBM

28. Select best evidence…
ACQUIRE/ COLLECT
Answering the Question…
Relevant?
Focused?
Appropriate Study Type?
Latest/Up to Date?
The BEST EVIDENCE is:
• External - from outside resources (researchers, experts)
• Current – not out of date, most recent
• High Quality - accurate, precise, effective, safe
• Patient focused - applicable and appropriate for your
individual patient

29. Is there a match?
ACQUIRE/ COLLECT
Cross-Sectional Study
Cohort Study
Randomized Controlled Trials
Qualitative Research
Patient Preference

30. FIVE STEPS TO FINDING THE BEST EVIDENCE
ACQUIRE/ COLLECT
IDENTIFY NEEDS: What type of information is needed?
IDENTIFY RESOURCES: Types, Availability, Timelines, Costs?
SEARCH & RETRIEVE: Use efficient strategies
REVIEW : Check quality and usefulness of info
INTERPRET: Help patient understand info, application
IDENTIFY YOUR RESOURCES
Colleagues
Consultation, Discussion
(Caution: Response may be an outdated “This is
what we do”)
Paper resources
books, reports, journals
Electronic databases
Health Literature Services
specialized librarians, staff
Review services, Abstract Services, etc.
ASK FOR HELP!
SPECIALIZED PERSONNEL
track down information, textbooks, articles, guidelines
may provide electronic search support or training
EXAMPLES
Medical Librarians
Medical Informatics Specialists
Specially trained staff member

31. Classify the Clinical Question into Domain
ACQUIRE/ COLLECT
• Therapy
• Randomized controlled trials or meta analyses
• Diagnosis
• Sensitivity and specificity, predictive value, diagnostic errors
• Prognosis
• Cohort studies, survival analysis
• Harm or Casualty
• Case control studies, cohort studies

32. What do we mean by Research Quality?
COLLECT/ APPRAISE
It’s All About “CONFIDENCE” 
The confidence that the study design, conduct and analysis has minimized biases in addressing the
research question
The better the quality, the higher is the likelihood that the results produced in the study are
credible
Validity
The degree to which the results of an
observation are correct for the
patients being studied.
Bias
A process that tends to produce results that
depart systematically from the true values
existing in the study population.
Fletcher et al, 1988; Murphy, 1976
Conscious Unconscious
Conflict of
Interest

33. All Evidences Are Not Created Equal
COLLECT/ APPRAISE
Journal name is NOT enough: Quality of
Medical Literature may wildly vary, even in
best of the journals
Quality of Medical Literature
Journal High Quality Articles
N Eng J Med 17%
Ann Intern Med 13%
JAMA 12%
BMJ 9%
Lancet 8%
Even in best of the journals, >80% of the articles
may not be of “high quality”
Then comes “Research Quality”

34. “LIMITERS”
COLLECT/ APPRAISE
Narrow search using “limiters”:
Examples of limiters:
• Date of publication
• Type of study
• Language
• Human
• Specific age group
• Full text online
• Sensitivity and Specificity
USE YOUR OWN FILTERS (EXPERTISE)
Validity
Can I trust this information?
Clinical Importance
If true, will the use of this information make an important difference?
Applicability
Can I use the information in this instance?

35. WHAT STUDY DESIGN FITS IT BEST?
COLLECT/ APPRAISE
EXPERIMENTAL TRIALS
(Answers questions of diagnosis or treatment)
• Randomized Controlled Trials (RCTs)
• Controlled studies
• Blinded vs Open
ETC.
OBSERVATIONAL STUDIES
• Descriptive reports
• Retrospective studies
• Cohort studies
• Case Control
ETC.
EXAMPLE
Randomized Controlled Trials (RCT)
“Gold Standard” of research
Ideal experimental design - Best design for TREATMENT questions
Must identify objective of treatment
(Ex: cure, prevent complication, palliation, reassurance)
Still not always the right intervention for individual patient at that particular time
and place

36. The Hawthorne Effect
APPRAISE
Outcomes can change

By virtue of doing the study

Irrespective of the intervention
The alteration of behavior by the subjects of a study due to their awareness of being observed

37. A Small Detour…
APPRAISE

38. Basic Research Principles
Study Designs
When was the study done
What was it’s duration
In which time direction was it headed…
RETROSPECTIVE or PROSPECTIVE
A: Time Factor
• What Year
• Which technology (test/drug/procedure etc.)
• Any associated social factor or historical event?
• Was the length of study suitable for the intended goal?
• Limited time study or ongoing?
• Was study completed? Stopped early?
Retrospective:
• PRO: May provide good direction for future study, “Hind Sight is 20/20”
• CON: Prone to Bias, A “Fishing Expedition” for positive results
Prospective:
• PRO: Lower risk of bias
• CON: May get faulty results based on incomplete data or insignificant
subgroups
(Example of Error: Untreated hypertension unlikely to cause cardiac event in child, so treatment is
unnecessary below age 18yrs)

39. Controlled vs. Uncontrolled Studies
Was there a similar study group
UNCONTROLLED STUDY
No comparison group
All subjects receive Experimental Intervention
“Trial and Error?” or “Before & After?”
Generally NOT accepted:
Potentially Dangerous and Flawed
Prone to BIAS!
Trial and Error: PROBLEMS
POSITIVE OUTCOME MAY BE DUE TO:
Other factors, Natural course of disease (some get better, some don’t!),
Spontaneous change of health, Placebo Effect, Hawthorne Effect
NEGATIVE OUTCOME, May be due to study treatment, Could be disastrous!
BEFORE or AFTER:
Benefits: Can answer some questions about: likelihood of response, adverse effect, etc.
Is VERY PATIENT-SPECIFIC!
But MAY BE ONLY OPTION in Rare conditions, Previously unknown conditions

40. UNCONTROLLED TRIALS: “TRIAL AND ERROR”
Example #1 Smallpox Vaccination
1. 1796: Edward Jenner inoculates 8yr-old James Phipps with cowpox virus from a
milkmaid’s hands. Child develops illness, recovers.
2. Two weeks later, inoculates same child with smallpox virus.
Child survives, no illness.
(Centuries later, smallpox eradicated!)
n=1
Example #2 Helicobacter pylori
History Time: China - powdered smallpox scabs blown into sinuses, pills made from cow fleas, swallowed, India and Istanbul - smallpox scabs applied to the scarified skin of healthy
(variolation), 1717 Lady Montague has son “ingrafted” by court physician, later tests orphans, prisoners
1982: Australian microbiologist Barry J. Marshall presents evidence showing a
possible infectious cause for gastric ulcers. Suggests they may be treatable with
antibiotics. Findings are met with disinterest and disbelief by medical
community. Lacks support for further study. 5 years later: Prepares a broth of
live organisms isolated from a gastric ulcer patient and drinks it. Becomes
violently ill, develops severe acute gastritis. Today, Antibiotics are used routinely
to cure some gastric ulcers!
n=1

41. Past  Future
Uncontrolled vs. Controlled Trials
May represent the ONLY treatment option for a new or rare
disease
Uncontrolled
Trials
Past  Future
STRONGLY PREFERRED! Reduces BIAS. Provides stronger results.
Controlled
Trials

42. Controlled Trials
Only the TEST group receives the Experimental Intervention
Control group may receive…
IMPORTANT
All other differences should be minimized or eliminated to
reduce potential BIAS

43. Randomized Controlled Trials:
The Gold Standard
THE FIRST RANDOMIZED CONTROLLED TRIAL
By Sir Austin Bradford Hill
1944 TUBERCULOSIS TREATMENT: Streptomycin vs Bedrest

44. Open vs. Blinded Study
Blinded
BLINDING
SINGLE BLINDED:
Pt unaware of what group s/he is in
DOUBLE BLINDED:
Pt and Investigator both unaware
OPEN LABEL:
Everyone is aware

45. Randomized vs. Non-randomized Trials
Non-Randomized
Randomized

46. Present  Future
Controlled + Randomized:
Randomized Controlled Trials: The Gold Standard

47. Other common studies
• Single Patient series of tests (n=1)
BENEFIT: Produces data most applicable to the individual patient
EXAMPLES: Trial of different medications and/or placebo for child reported to have ADHD symptoms that are not clinically apparent, Trial of
different analgesics for patient with chronic pain from a combination of diseases not previously studied
PATIENT Must be blinded, Must keep diary or complete questionnaire
PHYSICIAN May need to be blinded (enlist help of pharmacist!), Must treat patient as usual in all other respects

48. Other common studies
• Crossover Trials: One group: Multiple tests (best if participants are blinded)
Pros:
Lowers cost: All patients are in the experimentation group (fewer
subjects than RCTs)
Challenges:
MUST HAVE SHORT CARRYOVER EFFECT
MUST HAVE SHORT WASHOUT EFFECT
(OR WAIT A SUITABLY LONG WASHOUT TIME!)

49. Other common studies
• Case Control Studies: All about looking back

50. Other common studies: Cohort “Follow-up design)
TO INVESTIGATE ETIOLOGY OR HYPOTHETICAL CAUSE OF
DISEASE/OUTCOME
Present
Dialysis Pts
(example)
RISK FACTOR
Hgb <9
Measures future outcome for dialysis
pts w/o treatment of anemia

51. Other common studies

52. Problems of looking back

53. RANDOMIZED & CONTROLLED TRIAL (RCT)
Experimental
Intervention
Control Group
PROSPECTIVE
MAY BE BLINDED

54. START WITH YOUR TARGET POPULATION
Set CRITERIA for
INCLUSION / EXCLUSION
This will determine:
ELIGIBILITY at the start
VALIDITY at the end

55. START WITH YOUR TARGET POPULATION

56. ELIMINATE THOSE WHO DO NOT MEET THE CRITERIA

57. NEXT: GATHER A SAMPLE GROUP

58. THE SAMPLE GROUP WILL:
•Represent the target population
•Meet the criteria for inclusion / exclusion
SIDE NOTES…
Study should be approved by an Ethics
Committee
Informed consent should be obtained
from study participants

59. SAMPLE GROUP MAY BE SUBDIVIDED FURTHER
STRATIFICATION
Divide into subgroups based on important
similar characteristics
RANDOMIZATION
Divide into sub-groups based on
unknown confounders

60. STRATIFICATION
“important similar characteristics”
Examples:
• Male or Female
• Age
• Stage of illness
• Prior illness or treatment
• Hospital vs Office groups
• Comorbid condition
• Etc.

61. EXAMPLE OF
STRATIFICATION
FEMALE
MALE

62. RANDOMIZATION
“unknown confounders”
Examples:
• Postal code
• Month of birth
• Random number
• Etc.

63. EXAMPLE OF
RANDOMIZATION
DX IN JANUARY-JUNE
DX IN JULY-DECEMBER

64. Experimental
Intervention
Control Group
Next… Divide your sample group(s) into STUDY GROUPS
“Test Group”
“Baseline Group”
Receives Experimental
Intervention
“Baseline Group”
• Nothing
• Observation
• “Same” miscellaneous
intervention (non-
experimental)
• Placebo
• “Gold Standard” therapy -
especially if unethical to do
otherwise!
“Test Group”

65. ASSIGN PATIENTS TO STUDY GROUPS
Experimental
Intervention
Control Group
Use caution against bias!
Sample Group Study Groups

66. Experimental
Intervention
Control Group
STUDY INVESTIGATOR
 usually assigns patients to
study groups.
 usually has a personal
preference for the
treatment or patient
 might unconsciously “work
harder” to make the study
work with non-preferred
candidates
= POTENTIAL FOR BIAS

67. Experimental
Intervention
Control Group
Use random
separation
and assignment!
RANDOMIZED CONTROLLED TRIAL (RCT)

68. Experimental
Intervention
Control Group
RANDOMIZED CONTROLLED TRIAL (RCT)

69. RANDOMIZED CONTROLLED TRIAL (RCT)
Experimental
Intervention
Control Group
FUTURE
Present
Proceed with study

70. Experimental
Intervention
Control Group
RANDOMIZED CONTROLLED TRIAL (RCT)
EXPERIMENTAL EVENT RATE (EER)
CONTROL EVENT RATE
(CER)

71. Experimental
Intervention
Control Group
FUTURE
Present
RANDOMIZED CONTROLLED TRIAL (RCT)
EXPERIMENTAL EVENT RATE (EER)
CONTROL EVENT RATE
(CER)
“The Gold Standard”

72. Disadvantages of RCT
• Expensive
• large # pts needed
• Prolonged recruitment and follow-up time needed
• Funding difficult to obtain except w/support of sponsors (challenging!)

73. Detour Over 
APPRAISE

74. APPRAISE
Interpreting the evidence
How to read a paper… How to do the math…
Critical Appraisal
EVALUATE WRITTEN EVIDENCE FOR…
• Quality
• Usefulness
…BY ASSESSING
Validity
Reliability
Relevance
Clinical importance

75. APPRAISE Critical Appraisal: VALIDITY
• What was the original purpose of the study?
• When was it prepared?
• By whom? credentials? affiliations?
• Sample population: Did the subjects represent an
appropriate test group? How were they selected?
Were controls used? Were groups similar for
important prognostic characteristics?
• How was the information gathered and
processed? Were groups treated equally except
for trial therapy? Were appropriate criteria used
to measure results? Were criteria applied
rigorously?
• Was the study completed? (Or ended early for a specified reason?)
• Did the study account for all test subjects? Including subjects lost to
follow-up? Were ALL pts analyzed in their allocated groups? (ie:
INTENTION TO TREAT - not “completed treatment” analysis)
• Information: Does the paper support its claims? Is the information
accurately presented? Does it represent the truth?
• Results: Are the results believable? To what degree of confidence?
Opinions may vary on angiograms, EKGs, radiographs, pathology, PAP
tests, etc.
• Comprehensiveness; Size: Was it large enough to yield credible
results? Thoroughness: Was it complete enough? Duration: Was it
long enough?

76. APPRAISE Critical Appraisal: Reliability
Do we trust the information and results?
• APPROPRIATE TYPE OF STUDY
• REPRODUCEABILITY
• INTERPRETATION OF RESULTS
• BIAS
APPROPRIATE TYPE OF STUDY
Was the type of study design used proper for the question?
eg: RCT would be choice for questions on TREATMENT
Are the Measurements and Results reproducible?
Different determinations may be caused by:
• Variation in measurement methods
• Different interpretation of results
• Lack of agreement (eg. why BP results vary?)
Would the same results be obtained if patient is re-measured?
(with identical procedure)
• at another time?
• by another person?
Were any similar studies done?
Was the information comparable? Did the results agree?
INTERPRETATION OF RESULTS
Is there consistency among researchers?
Different determinations may be caused by:
• Variation in measurement methods
• Different interpretation of results
• Lack of agreement
Were any new questions or controversies raised by the study?
Is there any evidence of bias? (A dangerous pitfall)
PATIENTS/RESEARCHERS

77. APPRAISE BIAS
Patient Bias
Social Desirability Bias
A. Patient responds in the way they perceive as correct
to support doctor
to support a preconceived notion
B. Patient denies unhealthy behavior, gets misclassified
Eg: Smoker vs Non-smoker
Hawthorne Effect
People act differently when they know they are being watched.
Ex: Follow more careful diet when regular weigh-ins are scheduled
Authors must take steps to reduce this bias by treating all equally!
Eg: Weigh all patients with same frequency, even for group not on special diet

78. APPRAISE BIAS
Physician Bias
Who sponsored or funded the study?
Personal gain or loss from results?
Affiliates
Special interests
Conflict of interest
Biased goal?
To satisfy editors and reviewers… rather than solve real life clinical
problems
Criteria bias?
Risk-avoidance by researchers
(focus energy on topics that produce positive results)
Bias toward patients?
Sample selection criteria used (inclusive, exclusive)
Assignment to test group or control - Random? Blind?
Data collection methods used
applied similarly to all subjects, including controls?
starting point – prospective/retrospective, stage of patient?
Was assessment blind?
Data analysis
Were all potential subjects included in denominator or otherwise accounted?
Were they evaluated in originally designated group? (INTENTION TO TREAT)

79. APPRAISE Reducing/eliminating BIAS
CONDUCT BLIND STUDIES
Single
Double-blinded
USE INDEPENDENT OBSERVERS
When doctor and/or patient can not be blinded, blinded IO measures outcome
IO may even be unaware of study hypothesis
USE MULTIPLE OBSERVERS
Eg: Send subject slides to multiple pathologists for interpretation
ESTABLISH CLEAR STANDARDS
Exact methods to use to reduce variation in technique among researchers
Clear wording on surveys, etc
VALIDATING INSTRUMENTS
Repeat screening to check for correct answers on surveys
More frequent evaluations or surveys prevent guesstimates common to less frequent evaluation

80. APPRAISE next step
RELEVANCE
QUESTION: Is the report applicable to our…
Problem?
“Does it address the questions raised?”
Patient(s)?
“Will my patient respond like those in the study?”
Practice?
“Can it be done within my practice or circle?”
ARE THE STUDY PATIENTS
Comparable within the study? (similar traits, age, socioeconomic group,
stage of illness, treatment, etc.)
Comparable to your patient?
ARE THE STUDY PROFESSIONALS
Comparable to you?
(general/specialist, primary care/teaching hospital, etc.)
CLINICAL IMPORTANCE
Information can be true and interesting in theory, yet useless in clinical practice!
Is the information clinically important?
If yes, how important is it?
study design - See: Hierarchy of Evidence
weight of results

81. “value of study design to maximize weight, minimize bias”
APPRAISE Hierarchy of Evidence
• Systematic Review of all relevant RCTs
• At least one properly designed RCT
• Trials and case studies
• Well-designed Controlled Trial without Randomization
• Well designed Cohort or Case Control Studies, preferably from >1 center or group
• Multiple Time series with or without intervention
(Exception: Dramatic results in uncontrolled trials, such as introduction of PCN in the 1940s)
• Opinions of respected authorities, based on
• Clinical expertise
• Descriptive studies
• Reports of Expert Committees

82. Hierarchy of Evidence
APPRAISE
Experimental studies
Randomized controlled trials
Controlled Observational studies
Case-control studies
Uncontrolled Observational studies
Case series
Case reports
Sacks et al. Am J Med 1982;72:233-240;
Cook et al. Chest 1992;102:305s-311s;
Guyatt et al. JAMA 1993;270:2598-2601

83. RCT: Evaluation
APPRAISE
Were all clinically appropriate outcomes measured?
Did an ethics committee approve the study?
Any statistically significant results also clinically significant?
Any significant adverse reactions?
Was follow-up procedural analysis identical?
Was continuous data analysis vs end of trial data used?

84. Putting to Practice: Guidelines
APPLY
Putting to Practice
Levels of evidence
Type of evidence
Evidence obtained from at least one randomized controlled trial or from meta-analysis of randomized
controlled trials
Evidence obtained from at least one well-designed controlled study without randomization
Evidence obtained from well-designed non-experimental descriptive studies, such as comparative
studies, correlation studies and case control studies
Evidence obtained from expert committee reports or opinions and/or clinical experience of respected
authorities
Level
I
II
III
IV

85. Guidelines: Grading of Recommendations
APPLY
Putting to Practice
Recommendation
(Evidence level I)
Requires at least one randomized controlled trial as part of the body of literature of overall good
quality and consistency addressing the specific recommendation
(Evidence levels II, III)
Requires availability of well-conducted clinical studies but not randomised clinical trials on the topic of
recommendation
(Evidence level IV)
Requires evidence from expert committee reports or opinions and/or clinical experience of respected
authorities. Indicates absence of directly applicable studies of good quality
Grade
A
B
C

86. Implementation is the key
APPLYASSESS
Fancy trials and complex reports won’t help the patient… Treatment will!
Patient: “A methodological minefield”
Difficult time understanding background information
Personal priorities may conflict with yours

87. PATIENT
Recognize:
• Needs
• Choices
• Preferences
• Values
• Socioeconomic
concerns
INFORMATION
Help the patient to
understand and interpret
available information
PHYSICIAN
Respect the personal
priorities of the patient Help the patient to
negotiate a decision
on intervention,
treatment
CONFLICT?
SEPARATE
THE ISSUES!

88. PARADIGM SHIFT
APPLYASSESS
OLD: Doctor had authority
(despite the pile of unread journals!)
NEW: Current Best Evidence leads medical practice
but it MUST be individually applied
THE INDIVIDUAL PATIENT
Every patient is different. Treat YOURS and not others
The “ideal” course of action is not necessarily best for THIS patient.
EBM + Psychosocial factors =
THIS patient should be advised to take
THIS therapy at
THIS point in time.

89. Summarizing
Expertise,
Experience &
Pathophysiology
Clinical problem
Develop answerable
questions
Search and obtain
relevant articles
Critical appraisal of
evidence
Decision making
about diagnosis &
treatment

90. EBM: PROS, CONS and LIMITATIONS
PROS
• Clinicians update knowledge base routinely
• Improved understanding of research methods
• Physician becomes more critical in use of data
• Increased confidence in management decisions
• Increased computer literacy, data search technology
• Better reading habits
• Provides framework for group problem solving, team
generated practice
• Transforms weakness or paucity of knowledge into positive
change
OK to be uncertain/skeptical/flexible
• Integrates medical education, research and clinical expertise
• Can be learned by non-clinicians – other HCWs, patient
groups, purchasers, etc.
• Allows us to keep up with our better-educated patients!
• Increased contribution of junior MDs
• Increased patient benefit
• Better communication with patients re: rationale of
management decisions
• Promotes better and more appropriate use of limited
resources
• May reduce costs or medical care or practice by eliminating
outdated or unnecessary factors
• Can be learned at any stage of physician’s career

91. EBM: PROS, CONS and LIMITATIONS
CONS
• Time consuming
• Information overload
• Time to learn and practice
• Time may be needed for team conferencing, planning and
review
• Takes money to establish resource infrastructure – library,
office, computers, peripherals etc.
• Internet costs
• Programs, software information, CD-ROMS
• Subscription costs – online and paper resources
• May increase cost of care (but hopefully offset by
elimination of unnecessary medical interventions, tests,
journals, etc. – plus save time in getting proper intervention)
• Online references made to unavailable journals or
references
• Exposes gaps in the evidence (but provides ideas for
researchers!)
• Requires computer skills (but can be done with minimal
computer literacy and skill)
• May expose your current practice as obsolete or dangerous
(loss of authority and respect)

92. EBM: PROS, CONS and LIMITATIONS
LIMITATIONS
• Lack of evidence (shortage of studies)
• Difficulty applying evidence to care of a particular patient
• Lack of skills (search, appraise, etc.) Lack of time to learn and
practice EBM (promotes lifelong learning thru better focus)
• Lack of physician resources for instant access to evidence (EBM
has worldwide applicability)
• RESTRICTED AVAILABILITY OF LAB TESTS
• NON-TEXTBOOK CASE
co morbidity, additional risk factors
• AFFORDABILITY (Dr. & PT)“I can’t afford to practice EBM.”
• Language barriers – available evidence may be unreadable,
should be included
Physician attitude: Can be the greatest limitation!
“It decreases the importance of my clinical expertise”
(that’s a necessary component!)
“It only applies to those involved in research.”
(promotes cooperation among multiple physicians)
“It ignores patient values and preferences.”
“It’s just another cookbook approach to medicine.”
“It’s a poorly disguised way to cut medical costs.”
(cost of care may actually increase)
“It’s a way to ration care and resources.”
(Provides better utilization of avail resources)
DISAGREEMENT
Pt’s comfort, choice, acceptance, values, preferences Vs Dr’s
recommendations
DOES RISK OR SIDE EFFECTS OF TREATMENT OUTWEIGHT THE BENEFITS?

93. The unanswered question…
“DOES EBM REALLY MAKE A DIFFERENCE?”
• Effect of practicing EBM on patient outcome is actually unknown – no
studies done
EBM good based on population studies:
(ie: Pts who rec’d ___ generally fare better than those who don’t)