Ebm presentation

EVIDENCE BASED MEDICINE
A new approach to clinical care and research
Developed and presented by
Judy Tarselli, RN
Dubai, UAE
Karachi, Pakistan
October 2003
Organized by NKF cyberNephrology
University of Alberta, Canada
www.cyberNephrology.org
Special thanks to our sponsors Janssen-Cilag

PROGRAM OUTLINE
1. Definition of EBM
2. Basic Steps
3. Trials, Studies and Reports
4. Pros, Cons and Limitations
5. EBM in Developing Countries
6. EBM Library
7. Advanced EBM

A. Training, clinical experience and consultation
with other professionals
B. Convincing evidence (non-experimental) from
articles, case reports, product literature, etc.
C. Preferences of the patient
D. Active search of Randomized Controlled Trials,
Systematic Reviews, Meta-Analysis Reports
WHAT IS THE BASIS OF YOUR
MEDICAL PRACTICE?
(Check all that apply)

A. Training, clinical experience and consultation
with other professionals
B. Convincing evidence (non-experimental) from
articles, case reports, product literature, etc.
Systematic Reviews, Meta-Analysis Reports
MEDICAL PRACTICE?
EXCELLLENT!

BUT… Past knowledge and practice
might be outdated or inadequate
Graduate Medical School Practiced Physician

A. Training, clinical experience and consultation with
other professionals
B. Convincing evidence (non-experimental) from articles,
case reports, product literature, etc.
Systematic Reviews, Meta-Analysis reports
MEDICAL PRACTICE?
FANTASTIC!

BUT… This evidence may be biased, outdated,
incorrect, or not applicable to your patient
ARTICLES ADVERTISEMENTS
JOURNALS (1987 to present)

A. Training, clinical experience and consultation with other
professionals
MEDICAL PRACTICE?
WONDERFUL!
Mutual Respect +
Shared Goals =
Better Cooperation
and Compliance

The patient should be involved in
all important decisions
But this is NOT always an easy task!
And conflicts WILL occur!

But doctor, I DO want
to have children!
No salt?
Lose weight?
Forget it!
Just give me a pill!
I WON’T take that medicine…
The side effects are
INTOLERABLE!
And conflicts WILL occur!

Education about current alternatives and risks is often
needed… for both the Patient and the Doctor!
But doctor, I DO want
to have children!
No salt?
Lose weight?
Forget it!
Just give me a pill!
I WON’T take that medicine…
The side effects are
INTOLERABLE!

I’ll discuss those risks
with my husband.
Yes, I’d like to try that
new medication!
Wow…
I never knew that high
blood pressure could
be so dangerous at my
age!
Education about current alternatives and risks is often
needed… for both the Patient and the Doctor!

The patient’s preferences MUST be considered!
An important rule in Evidence Based Medicine…
It STARTS with the patient and ENDS with the patient.

professionals
WHAT IS THE BASIS OF YOUR MEDICAL
PRACTICE?
WOW!!! SUPERB!!!

In the practice of Evidence Based Medicine,
it is the physician’s duty to find the best and
most current information and apply it
judiciously for the benefit of the patient.

But… A practice based exclusively on science and math
is effective only if your patients are robots or clones!
Don’t forget to allow for individual human differences
and personal preferences!

professionals
MEDICAL PRACTICE?
If you checked all 4 items…

professionals
B. Convincing evidence (non-experimental) from articles, case reports,
product literature, etc.
D. Active search of Randomized Controlled Trials, Systematic
Reviews, Meta-Analysis reports
You are practicing EVIDENCE
BASED MEDICINE!
CONGRATULATIONS!

1. Definition of EBM
2. Basic Steps
3. Trials, Studies and Reports
4. Pros, Cons and Limitations
5. EBM in Developing Countries
6. EBM Library
7. Advanced EBM

“What is Evidence Based Medicine?”
“And where did it come from?”

A BRIEF HISTORY
1980’s: McMasters University in Ontario, Canada
Dr. David Sackett and colleagues proposed Evidence
Based Medicine (EBM) as a new way of teaching, learning
and practicing medicine.
Dr. Sackett defines EBM as:
“…The conscientious, explicit, and judicious use
of current best evidence in making decisions
about the care of individual patients.”

Evidence Based Medicine
It is a change in the way physicians practice medicine, teach and
learn, and handle research.
Clinical practice: Based on the best current evidence
(not necessarily on how it’s always been done)
Patient Care: Compassionate, patient-oriented
(less authoritarian)
Learning & Teaching: Problem-based, problem-solving
more investigative, less know-it-all-by-yesterday
Research: More stringent approach, better proof criteria
(more demanding of proof, less room for error)

PATIENT
PHYSICIAN
INFORMATION
Question
or
Problem
THREE MAJOR
COMPONENTS of
EBM

PATIENT
Values, Concerns Preferences,
Expectations
Life predicament
PHYSICIAN
Training & Experience
Current Expertise
Continued learning
Demand for proof
INFORMATION
Clinically relevant
Proven by research
Best up-to-date
evidence
EBM
THE ADDED DETAILS

PATIENT
Values, Preferences
Concerns, Expectations
Life predicament
PHYSICIAN
Training
Expertise
Continued Learning
Demand for proof
EBM
CHARITY
EBM is not a
required practice
(yet)
ENTHUSIASM
Challenge, Variety,
Change
HUMILITY
Non-authoritarian
practice
OPTIONAL
COMPONENTS
TO BE ADDED BY
THE PHYSICIAN
INFORMATION
Clinically relevant
Proven by research
Current, up to date

“Isn’t this the way
we have always
practiced medicine?”
“Aren’t these just the
same old ingredients
tossed into a new
recipe?”
When am I supposed to find
the time to do that?

THE FIVE BASIC STEPS OF EBM
1. Clinical Question
Patient-focused, problem-oriented
2. Find Best Evidence
Literary Search
3. Critical Appraisal
Evaluate evidence for quality and usefulness
4. Apply the Evidence
Implement useful findings in clinical practice
5. Evaluate
The information, intervention, and EBM process

The Clinical Question
The FIRST step
The HARDEST step
The MOST IMPORTANT step!

FACT: We all have informational needs!
That is not a problem!

Problems arise
• if we fail to recognize those needs
• if we fail to bridge the information gap
• if we fail to ask the right questions

And also for others
around you!
Lee, exactly how
much time did you
spend on that big
project?
Hmmm… Is he
about to give me a
BONUS?
Or is he about
to FIRE me?
It will make life
easier for you...
Asking good questions
is a skill to be learned.

A GOOD QUESTION…
• Is focused and relevant
• Provides clear
communication
• Clarifies your goal or need
• Will reduce the amount of
time needed to obtain the
answer
Lee, can you give me an
accounting of the extra time
you spent on that project so
that I can charge it back to
the client?
Oh sure! I’ll have it
on your desk by
tomorrow!

• Be specific
Identify the problem, clarifiy
the clinical issue
• Be answerable
through the literature
• Contain multiple aspects
(patient, options,
comparisons, etc)
WHEN PRACTICING EBM,
a good question must also:
ACTUAL CASE SCENARIO
Large cauc male, age 40
2mo ago: Presented with classic
nephrotic syndrome, significant
symptoms. Bx showed IgAN. Cr
1.4, incr to 2 range, now 1.6
Tried prednisone 60mg qd -
tolerated poorly w/tremors and
depression.
Needs new regimen, but others
are aimed more at nephritic IgA
rather than nephrotic syndrome.
Suggestions?
It should NOT involve a
question of Personal Preference
or Local Concern.

THE EVIDENCE BASED RESPONSE
Posted on Nephrol 4/13/03
“In the study below, proteinuria and renal
function improved on this combination:
Ballardie FW, Roberts IS. Controlled prospective
trial of prednisolone and cytotoxics in progressive
IgAN. J Am Soc Nephrol 2002 Jan….”
“I have patients on this regime who have
benefitted.”
Regards,
Dr. Paulose P. Thomas
Nephrologist - Belhoul Apollo Hospital, Dubai, UAE
Respondant recommends cyclophosphamide and
prednisolone (assuming secondary causes excluded) - a
combination that allows for lower dose prednisolone…

BACKGROUND and FOREGROUND QUESTIONS
(all part of EBM)
FOREGROUND QUESTIONS
BACKGROUND QUESTIONS
NEW POSSIBILITIES
INDEFINITE ANSWERS
“Where do we want to go,
and how else might
we get there?”
EXPERT
GRAD
STUDENT
“Where are we now?
And which way are we headed?”
BASIC & CONCRETE

BACKGROUND QUESTIONS
BASIC & CONCRETE
1. QUESTION
• Who, What, Where, When, Why, How
2. VERB
• is, causes, does, treats, reduces, cures, prevents, affects
3. GENERAL KNOWLEDGE ABOUT DISORDER
clinical manifestations of disease, patient findings, differential
diagnosis, etiology, patient experience, comorbid condition,
screening and diagnostic tests, prognosis, therapy, risk factors,
etc.
EXPERT
GRAD
STUDENT

FOREGROUND QUESTIONS
NEW POSSIBILITIES
INDEFINITE ANSWERS
PT AND/OR PROBLEM Differential dx, Unusual presentation, uncertain
etiology, pt’s prior experience, comorbid conditions
INTERVENTION Exposure, test. Prognostic factor,
treatment, pt perception, etc.
COMPARISON INTERVENTION
OUTCOMES
EXPERT
GRAD
STUDENT

EBM QUESTION: Should include multiple factors
(Examples)
P PATIENT type of patient or population
Ex: 47 yr male w/DM2 and cellulitis toe, 25 yr female w/DVT and chest pain
E EXPOSURE environmental, personal, biological
Ex: TB, tobacco, drug, diet, pregnancy or menopause, MRSA, allergy
I INTERVENTION clinical intervention
Ex: medication, procedure, test, surgery, radiation, drug, vaccine
C COMPARISON compare alternative treatment
Ex: other prior, new or existing therapy
O OUTCOME clinical outcome of interest
Ex: Reduced death rate in 5 yrs, decreased infections, fewer hospitalizations

FRAMING THE QUESTION (Example: PICO)
ELEMENT PROMPTS THE QUESTION:
Patient How would I describe a group of patients similar to mine?
Intervention What main action am I considering?
Comparison What is/are the other options?
Outcome What do I (or the patient) want to happen (or not happen)?
Example:
P: In kids under age 12 with poorly controlled asthma on metered
dose inhaled steroids…
I: would the addition of salmetrol to the current therapy
C: compared to increasing the dose of current steroid
O: lead to better control of symptoms without increasing side effects?

CATEGORY OF QUESTION
MAJOR CATEGORIES
1. Diagnosis
2. Prognosis
3. Therapy/ Treatment PICO
4. Harm (iatrogenic, other) PEO
MISCELLANEOUS
• Quality of care
• Health economics
• Office Management
• Etc.

THE PATIENT’S QUESTIONS
Must be considered!
Often QUALITATIVE (not based on measureable outcomes)
Feelings, ideas, experiences, preferences, concerns, fears, beliefs,
ethnicity
Usually based on LIMITED BACKGROUND
Perception of problem
Self-diagnosis
Treatment wanted or needed
Alternatives (read, heard, considered, tried)
What is the patient hoping to avoid?
What benefits does the patient want or need most?
Etc.

QUANTITATIVE: “Solid Evidence”
• Measurable answer or response
• Necessary for scientific study
• Necessary for the practice of EBM
QUALITATIVE: “Quality of Life”
• “Fuzzy” data - Impact on daily life, work, family, etc.
• May be very important and influential to decisions –
especially for the patient
• Creates added challenge or twist to practice of EBM
QUANTITATIVE vs QUALITATIVE QUESTIONS

QUALY: QUALITY ADJUSTED LIFE YEAR

Find the Best Evidence
“The Literary Search”
HINT: If your desk looks like this, it’s probably the
LAST place you should start looking!

Find the Best Evidence
“The Literary Search”
The BEST EVIDENCE is:
External - from outside resources (researchers, experts)
Current – not out of date, most recent
High Quality - accurate, precise, effective, safe
Patient focused - applicable and appropriate for your individual
patient

FIVE STEPS TO FINDING THE BEST EVIDENCE
1. IDENTIFY NEEDS: What type of information is needed?
2. IDENTIFY RESOURCES: Types, Availability, Timeliness,Costs?
3. SEARCH & RETRIEVE: Use efficient strategies
4. REVIEW : Check quality and usefulness of info
5. INTERPRET: Help patient understand info, application

WHAT TYPE OF INFORMATION IS NEEDED?
WHAT CATEGORY IS THE QUESTION?
• Diagnosis
• Prognosis
• Therapy
• Harm

WHAT STUDY DESIGN FITS IT BEST?
There are MANY study designs!
EXPERIMENTAL TRIALS
(Answers questions of diagnosis or treatment)
Randomized Controlled Trials (RCTs)
Controlled studies
Blinded vs Open
ETC.
OBSERVATIONAL STUDIES
Descriptive reports
Retrospective studies
Cohort studies
Case Control
ETC.

EXAMPLE
Randomized Controlled Trials (RCT)
“Gold Standard” of research
Ideal experimental design - Best design for TREATMENT questions
Must identify objective of treatment
(Ex: cure, prevent complication, palliation, reassurance)
Still not always the right intervention for individual patient at that particular time and
place

What type of evidence best addresses the question, problem or issue?
CLINICAL PRACTICE APPROPRIATE DESIGN FOR CLINICAL RESEARCH
Diagnosis, Dx testing Cross-sectional study – not randomized trial
Prognosis Follow-up studies of patients evaluated at same early point of illness
Therapy, treatment RCT or Systematic review of multiple RCTs must be used
Avoid non-experimental approaches to avoid false conclusions about efficacy
Exceptions:
When treatment may be successful in an otherwise fatal condition
When no studies are available (rare conditions, new treatments, etc.)
Harm RCT, Cohort, Case-control
OTHER INFORMATIONAL
Explore hypothesis Qualitative research
History-taking Case control study
Individual trial & error n of 1 trial
Following clinical course Cohort study
Recordkeeping Systematic registry-based (computer supported) research
Quality of Care research Individual peer review, Process Evaluation
MISCELLANEOUS Basic Science, Genetics, Immunology, etc.

WHAT FORM OF INFORMATION?
Case report
Controlled Trial
Systematic review
Meta-analysis
Clinical guidelines
etc.

LITERARY SEARCH: NEXT STEP
IDENTIFY YOUR RESOURCES
Colleagues
Consultation, Discussion
(Caution: Response may be an outdated “This is what we do”)
Paper resources
books, reports, journals
Electronic databases
Health Literature Services
specialized librarians, staff
Review services, Abstract Services, etc.

SEARCH AND RETREIVE THE BEST EVIDENCE
Learn and Practice various SEARCH STRATEGIES:
• To find useful information quickly
• To eliminate irrelevant, inappropriate or weak information
Try to develop the habit of learning as you go;
Not just in lengthy formal sessions!

LITERARY SEARCH STRATEGY
ASK FOR HELP!
SPECIALIZED PERSONNEL
• track down information, textbooks, articles,
guidelines
• may provide electronic search support or training
EXAMPLES
• Medical Librarians
• Medical Informatics Specialists
• Specially trained staff member

LITERARY RESOURCES
• TEXTBOOKS (caution – most obsolete!)
• Traditional
• Evidence Based
• JOURNALS (may be outdated)
• REVIEW ARTICLES (summaries, abstracts)
• SYSTEMATIC REVIEWS (prepared in systematic, rigorous
manner) Ex: Cochrane Collection
• META-ANALYSIS
• CLINICAL PRACTICE GUIDELINES
Summarized and easily digestible information

ELECTRONIC RESOURCES, DATABASES, INTERNET
Bibliographic Database
Example: Medline, PubMed
Medical Information Services: Medscape, HDCN
Review Services
Subjective
Systematic Reviews
Meta-analysis
Examples:
• Cochrane,
• Best Evidence,
• Up to Date

MORE GREAT INTERNET RESOURCES
Websites
cyberNephrology, National Kidney Foundation. NIDDK,
American Heart Association, American Cancer Society.
National Institutes of Health, etc
Listserve Discussion Groups
CyberNephrology, C-span, etc.
Specialty Electronic Databases
Psyclit
CancerLit
CINAHL
(allied health and nursing journals)
Etc

OTHER RESOURCES
Tapes
Videos
CD-ROMs
Specialty seminars
Product information and comparisons

A closer look at some Internet Resources…

MEDLINE
WHAT IS IT?
Searchable database of medical information compiled by National Library of
Medicine in US 1966-present
Catalogs articles from approx 4000 world journals (of estimated 12-15k total)
SEARCH METHODS
Any word or words (title, abstract, content, author name, institution, etc.)
Medical Subject Heading (MeSH) terms
A restricted thesaurus of medical titles
Articles categorized by most specific possible MeSH heading

COST: FREE!
Or may subscribe to companies with specialized search strategies:
• Ovid Technologies (ovid)
• Silver Platter Information (WinSPIRS)
BENEFITS
Free
Vast database
LIMITATIONS
Not all articles are indexed on Medline (only 1/3 of approx 10 million!)
Much material listed and described on Medline can only be accessed through
journal article

MEDLINE: ELECTRONIC SEARCH STRATEGIES
Search through “Clinical Queries” service of PubMed
http://www.ncbi.nlm.nih.gov/clinical.html
Medical Subject Headings (MeSH)
Search filters
Search by a text word can supplement a MeSH search
Boolean search: “and”, “not”, etc.
To increase sensitivity
• use “explode” command
• avoid using subheadings
Online Tutorial is available!

COCHRANE LIBRARY
Cochrane Database of Systematic Reviews
-systematically compiled reviews of intervention
Cochrane Controlled Trials Register
-citations of controlled trials identified anywhere in the world
Cochrane Review Methodology Database
-methodological papers relating to systematic reviews
Etc.

BEST EVIDENCE
Electronic version of two publications:
• Evidence Based Medicine
• American College of Physicians Journal Club
Covers broad topics of information

CRITICAL APPRAISAL
Interpreting the evidence
• How to read a paper
• How to do the math

CRITICAL APPRAISAL
IMPORTANT!
You do NOT have to become a researcher,
epidemiologist, or statistician to practice EBM.
Focus on how to USE research reports –
not on how to generate them!

HOWEVER…
You must have a solid understanding of
basic research principles and
study designs in order to understand
and interpret the evidence!
CRITICAL APPRAISAL

TYPES OF STUDIES AND REPORTS
Randomized Controlled Trial - “The Gold Standard”
Systematic review
Meta-analysis
Retroactive vs Prospective
Incidence
Prevalence
Case Control
Cohort (Follow-up)
Cross-sectional
Ecologic
Longitudinal
Experimental
Blinded vs Open
Qualitative Screening

BASIC RESEARCH PRINCIPLES
STUDY DESIGNS

THE TIME FACTOR
When was the study done?
In what time direction is it headed?
What was its duration?
RETROSPECTIVE PROSPECTIVE

THE TIME FACTOR
When was the study done?
What year?
What technology? (ie: test, drug, equipment, procedure)
Any associated social factor or historical event?

THE TIME FACTOR
What was the Study Duration?
Was it an appropriate length of time for the
intended goal?
Limited time study or ongoing?
Was study completed? Stopped early?

“LOOKING BACK”
Historical Review or
Investigation
“LOOKING FORWARD”
Future Results
The Great Unknown
PRESENT
PAST FUTURE
In what direction is it headed?

PRO
•May provide good
direction for future study
“Hind Sight is 20/20”
CON:
•Prone to Bias
•A“Fishing Expedition” for
positive results
PRO
•Lower risk of bias
CON:
May get faulty results based
on incomplete data or
insignificant subgroups
(Example of Error: Untreated
hypertension unlikely to cause
cardiac event in child, so treatment
is unnecessary below age 18yrs)
PRESENT
In what direction is it headed?

“Was there a similar comparison group?”

Experimental
Intervention
No comparison group
All subjects receive Experimental Intervention

Experimental
Intervention
NO EVENT
OUTCOME
EVENT
“Trial and Error?” or “Before & After?”
UNCONTROLLED STUDIES

PROBLEMS
POSITIVE OUTCOME MAY BE DUE
TO:
•Other factors
•Natural course of disease (some
get better, some don’t!)
•Spontaneous change of health
•Placebo Effect
•Hawthorne Effect
NEGATIVE OUTCOME
May be due to study treatment.
Could be disastrous!
BENEFITS
Can answer some questions
about:
•likelihood of response
•adverse effect, etc.
VERY PATIENT-SPECIFIC!
MAY BE ONLY OPTION
Rare conditions
Previously unknown conditions
“Trial and Error” “Before & After”
UNCONTROLLED STUDIES
Generally NOT accepted:
Potentially Dangerous and Flawed
Prone to BIAS!
“Traditional Study Method”
May produce strong results

SMALLPOX
VACCINATION
SMALLPOX VACCINE
1. 1796: Edward Jenner inoculates 8yr-old James Phipps with cowpox virus
from a milkmaid’s hands.
Child develops illness, recovers.
2. Two weeks later, inoculates same child with smallpox virus.
Child survives, no illness.
(Centuries later, smallpox eradicated!)
n=1
GOOD!
Resistant to
Cowpox and
Smallpox
(NO DISEASE
OUTCOME)
James Phipps,
age 8 years
Example#1
UNCONTROLLED TRIALS: “TRIAL AND ERROR”

Drinks culture of
H.pylori
HELICOBACTER PYLORI - GASTRIC ULCERS
1982: Australian microbiologist Barry J. Marshall presents evidence showing a
possible infectious cause for gastric ulcers. Suggests they may be treatable with
antibiotics.
Findings are met with disinterest and disbelief by medical community. Lacks
support for further study.
5 years later: Prepares a broth of live organisms isolated from a gastric ulcer
patient and drinks it. Becomes violently ill, develops severe acute gastritis.
1990’s Antibiotics are used routinely to cure some gastric ulcers!
Example #2 NO
OUTCOME
SEVERE
GASTRITIS
n=1
UNCONTROLLED TRIALS: “TRIAL AND ERROR”
Dr. Marshall
Microbiologist

UNCONTROLLED TRIAL
Experimental
Intervention
Present
FUTURE
May represent the ONLY
treatment option for a new or rare
disease
DIED
RECOVERED

Experimental
Intervention
Control
Group
STRONGLY PREFERRED! Reduces BIAS. Provides stronger results.

Nothing
Placebo
Observation only
ExperimentalI
ntervention
Control group may receive…
Only the TEST group receives the Experimental Intervention
CONTROLLED STUDY
Other
IMPORTANT
All other differences
should be minimized or
eliminated to reduce
potential BIAS
Gold Standard
Treatment

Experimental
Intervention
Control
Group
RANDOMIZED CONTROLLED TRIAL (RCT)
“The Gold
Standard”

1944 TUBERCULOSIS TREATMENT: Streptomycin vs Bedrest
Streptomycin
(n=50)
Bedrest
(n=50)
THE FIRST RANDOMIZED CONTROLLED TRIAL
By Sir Austin Bradford Hill
(BLINDED)

Experimental
Intervention
Control
Group
OPEN vs BLINDED STUDIES
OPEN

BLINDED
TRIAL
BLINDED
TRIAL
BLINDED
OPEN vs BLINDED STUDIES

BLINDING
SINGLE BLINDED:
Pt unaware of what group s/he is in
DOUBLE BLINDED:
Pt and MD unaware
OPEN LABEL:
Everyone is aware

RANDOMIZED vs NON-RANDOMIZED TRIALS
Experimental
Intervention
Control
Group
How is this
group divided?

NON-RANDOMIZED
Experimental
Intervention
Control
Group
Assigned to
groups, usually
by the
researcher
Potential for RESEARCHER BIAS!

RANDOMIZED
Experimental
Intervention
Control
Group
Random method of
assignment used
Maximizes “sameness,” Eliminates BIAS!

(EXPERIMENTAL TRIAL)
Experimental
Intervention
Control
Group
Present
FUTURE
“The Gold Standard”

Experimental
Intervention
Trial of Medicine
1
Or placebo
TRIAL SERIES FOR INDIVIDUAL PATIENT
n=1
One patient, series of tests
Experimental
Intervention
Trial of Medicine
2
Or placebo
GOOD GOOD
NO CHANGE
OR BAD
NO CHANGE
OR BAD

Why a TRIAL SERIES for one patient?
EXAMPLES:
Trial of different medications and/or placebo for child reported
to have ADHD symptoms that are not clinically apparent
Trial of different analgesics for patient with chronic pain from
a combination of diseases not previously studied
PATIENT
•Must be blinded
•Must keep diary or complete
questionnaire
PHYSICIAN
•May need to be blinded (enlist help
of pharmacist!)
•Must treat patient as usual in all
other respects
BENEFIT
Produces data most applicable to the individual patient

Intervention
A
ONE GROUP, MULTIPLE TESTS
CROSSOVER TRIALS
Intervention
B
Intervention
B
Intervention
A
ASSESS
OUTCOMES #1
ASSESS
OUTCOMES #2
COMPARE OUTCOMES
(Best if participants are blinded)

Intervention
A
CROSSOVER TRIALS
Intervention
B
Intervention
B
Intervention
A
ASSESS
OUTCOMES #1
ASSESS
OUTCOMES #2
Fewer participants needed than a RCT!
Lower costs
All are in experimental group
PROS & CONS

Intervention
A
CROSSOVER TRIALS
Intervention
B
Intervention
B
Intervention
A
ASSESS
OUTCOMES #1
ASSESS
OUTCOMES #2
MUST HAVE SHORT CARRYOVER EFFECT
MUST HAVE SHORT WASHOUT EFFECT
(OR WAIT A SUITABLY LONG WASHOUT TIME!)
PROS & CONS

CASE CONTROL
Present
RISK FACTOR?
(PAST)
(“A LOOK BACK”)

CASE CONTROL
Present
NEVER
SMOKED
(PAST)
(“A LOOK BACK”)
SMOKER
LUNG CANCER
HEALTHY
RISK FACTOR

CASE CONTROL
Present
NORMAL
WEIGHT
(“A LOOK BACK”)
OBESITY
DM TYPE II
NON-DIABETIC
RISK FACTOR

COHORT
IS RISK
FACTOR
PRESENT?
Future Outcome
“FOLLOWUP DESIGN”
(Exclude those
with outcome
already!)

COHORT
IS RISK
FACTOR
PRESENT?
Future Outcome
“FOLLOWUP DESIGN”
Present
TO INVESTIGATE ETIOLOGY
OR HYPOTHETICAL CAUSE
OF DISEASE/OUTCOME

COHORT
Present
RISK FACTOR
Hgb <9
EXAMPLE
DIALYSIS PATIENTS
Measures future outcome for
dialysis pts w/o treatment of anemia

CROSS SECTIONAL DESIGN
? Cause ? Risk factors
A look back

CROSS SECTIONAL DESIGN
INFANT
DEATHS
SIDS DEATHS
OTHER CAUSES
RISK = SLEEP PRONE

NO
CONTROL
OVER
CONTROL
GROUP
VARIATION IN
TREATMENT
OR METHOD
NON-SIMILAR
CONDITIONS
Social
Personal
Comorbid conditions
Other treatments
Etc.
Not usually
accepted by
medical journals
(accepted in
popular press,
not reviewed)
CURRENT
GROUP OF
PATIENTS
Problems of looking back

RANDOMIZED & CONTROLLED TRIAL (RCT)
Experimental
Intervention
Control
Group
PROSPECTIVE
MAY BE
BLINDED

START WITH YOUR TARGET POPULATION

START WITH YOUR TARGET POPULATION
Set CRITERIA for
INCLUSION / EXCLUSION
This will determine:
ELIGIBILITY at the start
VALIDITY at the end

ELIMINATE THOSE WHO DO NOT MEET THE CRITERIA

THE SAMPLE GROUP WILL:
•Represent the target population
•Meet the criteria for inclusion / exclusion
SIDE NOTES…
Study should be approved by an
Ethics Committee
Informed consent should be
obtained from study participants

SAMPLE GROUP MAY BE SUBDIVIDED FURTHER
STRATIFICATION
Divide into subgroups based on
important similar characteristics
RANDOMIZATION
Divide into sub-groups based on
unknown confounders

STRATIFICATION
“important similar characteristics”
Examples:
• Male or Female
• Age
• Stage of illness
• Prior illness or treatment
• Hospital vs Office groups
• Comorbid condition
• Etc.

EXAMPLE OF
STRATIFICATION
FEMALE
MALE

RANDOMIZATION
“unknown confounders”
Examples:
• Postal code
• Month of birth
• Random number
• Etc.

EXAMPLE OF
RANDOMIZATION
DX IN JANUARY-JUNE
DX IN JULY-DECEMBER

Experimental
Intervention
Control
Group
Next… Divide your sample group(s) into STUDY GROUPS
“Test Group”
“Baseline Group”

Experimental
Intervention
Control
Group
Next… Divide your sample group(s) into STUDY GROUPS
Receives Experimental
Intervention
“Baseline Group”
• Nothing
• Observation
• “Same” miscellaneous
intervention (non-
experimental)
• Placebo
• “Gold Standard” therapy -
especially if unethical to do
otherwise!
“Test Group”

ASSIGN PATIENTS TO STUDY GROUPS
Experimental
Intervention
Control
Group
Use caution against bias!
Sample Group Study Groups

Experimental
Intervention
Control
Group
STUDY INVESTIGATOR
 usually assigns
patients to study
groups.
 usually has a
personal preference
for the treatment or
patient
 might unconsciously
“work harder” to
make the study work
with non-preferred
candidates
= POTENTIAL FOR
BIAS

Experimental
Intervention
Control
Group
Use random
separation
and assignment!

Experimental
Intervention
Control
Group

Experimental
Intervention
Control
Group
FUTURE
Present
Proceed with study

Experimental
Intervention
Control
Group
EXPERIMENTAL EVENT
RATE (EER)
CONTROL
EVENT RATE
(CER)

Experimental
Intervention
Control
Group
FUTURE
Present
EXPERIMENTAL EVENT
RATE (EER)
CONTROL
EVENT RATE
(CER)
“The Gold
Standard”

Disadvantages of RCT
Expensive
large # pts needed
Prolonged recruitment and follow-up time needed
Funding difficult to obtain except w/support of
pharmaceutical companies (problematic!)

CRITICAL APPRAISAL

EVALUATE WRITTEN EVIDENCE FOR…
Quality
Usefulness
…BY ASSESSING
Validity
Reliability
Relevance
Clinical importance

Critical Appraisal: VALIDITY
What was the original purpose of the study?
When was it prepared?
By whom?
• credentials?
• affiliations?
Sample population
Did the subjects represent an appropriate test group?
How were they selected?
Were controls used?
Were groups similar for important prognostic characteristics?

VALIDITY
How was the information gathered and processed?
Were groups treated equally except for trial therapy?
Were appropriate criteria used to measure results?
Were criteria applied rigorously?
Was the study completed?
(Or ended early for a specified reason?)
Did the study account for all test subjects?
Including subjects lost to follow-up?
Were ALL pts analyzed in their allocated groups?
(ie: INTENTION TO TREAT - not “completed treatment” analysis)

VALIDITY
Information
Does the paper support its claims?
Is the information accurately presented?
Does it represent the truth?
Results
Are the results believable?
To what degree of confidence?
Ex: Disagreement is not uncommon on angiograms, EKGs, radiographs, pathology, PAP
tests, etc.

VALIDITY
Comprehensiveness
Size: Was it large enough to yield credible results?
Thoroughness: Was it complete enough?
Duration: Was it long enough?

CRITICAL APPRAISAL: RELIABILITY
Do we trust the information and results?
1. APPROPRIATE TYPE OF STUDY
2. REPRODUCEABILITY
3. INTERPRETATION OF RESULTS
4. BIAS

RELIABILITY
APPROPRIATE TYPE OF STUDY
Was the type of study design used proper for the question?
Example:
RCT would be choice for questions on TREATMENT

RELIABILITY
Are the Measurements and Results reproducibile?
Different determinations may be caused by:
• Variation in measurement methods
• Different interpretation of results
• Lack of agreement
Example:
BP checks on same patient may vary. Are differences result of pt factor, examiner
factor, treatment factor, normal variance
Would the same results be obtained if patient is re-measured?
(with identical procedure)
• at another time?
• by another person?
Were any similar studies done?
• Was the information comparable?
• Did the results agree?

RELIABILITY
INTERPRETATION OF RESULTS
Is there consistency among researchers?
Different determinations may be caused by:
• Variation in measurement methods
• Different interpretation of results
• Lack of agreement
EXAMPLE:
BANFF CONFERENCE - Setting standards in Transplant Pathology
established by Kim Solez, MD
Were any new questions or controversies raised by the study?

RELIABILITY
IS THERE ANY EVIDENCE OF BIAS?
A dangerous pitfall!
• PATIENTS
• RESEARCHERS

PATIENT BIAS
Social Desirability Bias
Patient responds in the way
they perceive as correct
• to support MD
• to support a preconceived
notion (ie: foods vs ADD)
Patient denies unhealthy
behavior, gets misclassified
Ex: Smoker vs Non-smoker

PATIENT BIAS
Hawthorne Effect
Authors must take steps to
reduce this bias by treating
all equally!
Ex: Weigh all patients with same
frequency, even for group not on
special diet
People act differently when
they know they are being
watched.
Ex: Follow more careful diet
when regular weigh-ins are
scheduled

RESEARCHER BIAS
Who sponsored or funded the study?
Personal gain or loss from results?
Affiliates
Special interests
Conflict of interest
Biased goal?
To satisfy editors and reviewers… rather than solve
real life clinical problems

Criteria bias?
Risk-avoidance by researchers
(will focus energy on topics that produce positive results)
Bias toward patients?
Sample selection criteria used (inclusive, exclusive)
Assignment to test group or control - Random? Blind?
RESEARCHER BIAS

Data collection methods used
• applied similarly to all subjects, including controls?
• starting point – prospective/retrospective, stage of patient?
• Was assessment blind?
Data analysis
• Were all potential subjects included in denominator or otherwise
accounted?
• Were they evaluated in originally designated group?
(INTENTION TO TREAT)
RESEARCHER BIAS

REDUCING OR ELIMINATING BIAS AND ERROR
CONDUCT BLIND STUDIES
• Single
• Double-blinded
USE INDEPENDENT OBSERVERS
• When doctor and/or patient can not be blinded, blinded IO measures outcome
• IO may even be unaware of study hypothesis
USE MULTIPLE OBSERVERS
Ex: Send subject slides to multiple pathologists for interpretation
ESTABLISH CLEAR STANDARDS
• Exact methods to use to reduce variation in technique among researchers
• Clear wording on surveys, etc
VALIDATING INSTRUMENTS
• Repeat screening to check for correct answers on surveys
• More frequent evaluations or surveys prevent guesstimates common to less
frequent evaluation

NEXT STEP IN CRITICAL APPRAISAL:
RELEVANCE
QUESTION: Is the report applicable to our…
Problem?
“Does it address the questions raised?”
Patient(s)?
“Will my patient respond like those in the study?”
Practice?
“Can it be done within my practice or circle?”

ARE THE STUDY PATIENTS
• Comparable within the study? (similar traits, age,
socioeconomic group, stage of illness, treatment, etc.)
• Comparable to your patient?
ARE THE STUDY PROFESSIONALS
• Comparable to you?
(general/specialist, primary care/teaching hospital, etc.)

NEXT STEP in CRITICAL APPRAISAL
CLINICAL IMPORTANCE
Information can be true and interesting in theory,
yet useless in clinical practice!
1. Is the information clinically important?
2. If yes, how important is it?
• study design - See: Hierarchy of Evidence
• weight of results

HEIERARCHY OF EVIDENCE
(value of study design to maximize wt, minimize bias)
1. Systematic Review of all relevant RCTs
2. At least one properly designed RCT
3. Trials and case studies
4. Well-designed Controlled Trial without Randomization
5. Well designed Cohort or Case Control Studies, preferably from >1
centre or group
6. Multiple Time series with or without intervention
7. (Exception: Dramatic results in uncontrolled trials, such as
introduction of PCN in the 1940s)
8. Opinions of respected authorities, based on
9. Clinical expertise
10. Descriptive studies
11. Reports of Expert Committees

Evaluation of RCT
Were all clinically appropriate outcomes measured?
Did an ethics committee approve the study?
Any statistically significant results also clinically significant?
Any significant adverse reactions?
Was follow-up procedural analysis identical?
Was continuous data analysis vs end of trial data used?

HOW TO DO THE MATH
Incidence
Prevalence
Statistical Formulas
+/- Predictive Values - Probability - The p value
Relative Risk
Risk Reduction
Odds Ratios
NNT (Number Needed to Treat) – Risk Reduction
Confidence Intervals
Sensitivity and Specificity
Regression Analysis
Subgroup Analysis
Health Status Evaluation
Health Economics

ACCOUNT FOR ALL even if
•Non-compliant
•Lost to follow-up
Analyze as a member of the originally assigned group!
Analysis SHOULD BE BASED ON
• INTENTION TO TREAT
• NOT on “completed treatment” analysis
OUTCOMES: NOT “STUDY FAILURES”
OUTCOMES RELATE TO EVERYDAY CLINICAL PRACTICE, including…
• Deaths
• Poor compliance
• Wrong treatment received
• Lost to follow-up
• Etc.

INCIDENCE & PREVALENCE
NEPHROL, a service of NKF cyberNephrology
7/10/03 10:17:12AM
Dear Nephrolers,
I would like to know how to calculate incidence and
prevalence of B and C virus in HD.
Thank you in advance.
Mario Cuba, MD
Servicio de Nefrologia
Hospital Lucia Iniguez Landin
Holguin, Cuba

INCIDENCE & PREVALENCE
Response from Michel Jadoul, MD
NEPHROL, a service of NKF cyberNephrology
Prevalence: total number of positive patients divided by total
number of patients: 20+/200=10%
Incidence: number of new positive cases/total number of
cases negative at start of period (e.g; year)/period (year?)
thus : for instance 2 new positive cases /100 negative cases
at start of year=2%/year.
M.Jadoul, M.D.

PREVALENCE
B
B
B
B
B
B
B
B
B
B
10 cases Hep B in 100 patients
10 / 100 = 0.10
PREVALENCE = 10%

INCIDENCE
B
B
B
B
B
B
B
B
B
B
B
B B
B
B
1 year period
90 HBsA(-) at start of study
5 new cases
5 / 90 = 0.06
INCIDENCE = 6% per year

B
B
B
B
B
B
B
B
B
B
B
B B
B
B RECALCULATED PREVALENCE
16 cases Hep B in 100 patients
16 / 100 = 0.16
NEW PREVALENCE = 16%

COMPARISON STUDIES: NEW DIAGNOSTIC TESTS
RESULTS OF GOLD STANDARD TEST
EXPERIMENTAL
TEST
POSITIVE
NEGATIVE
DISEASE
PRESENT
NO DISEASE
TRUE (+)
a
FALSE (+)
b
FALSE (-)
c
TRUE (-)
d
COMPARING A NEW TEST AGAINST THE GOLD STANDARD TEST

ACCURACY OF TEST - COMPARE TO GOLD STANDARD
What is the usefulness of the test in various groups and subgroups of pts?
RESULTS OF GOLD STANDARD TEST
EXPERIMENTAL
TEST
TEST
POSITIVE
a + b
TEST
NEGATIVE
c + d
DISEASE
PRESENT
a + c
NO DISEASE
PRESENT
b + d
TRUE (+)
a
FALSE (+)
b
FALSE (-)
c
TRUE (-)
d
TESTS ARE RARELY 100%
ACCURATE
THEY MUST BE COMPARED
AGAINST THE
GOLD STANDARD

RESULTS OF GOLD
STANDARD TEST
EXPERIMENTAL
TEST
TEST
POSITIVE
TEST
NEGATIVE
DISEASE
PRESENT
DISEASE NOT
PRESENT
TRUE (+)
a
FALSE (+)
b
FALSE (-)
c
TRUE (-)
d
TESTS ARE RARELY
100% ACCURATE
THEY MUST BE
COMPARED AGAINST
THE GOLD STANDARD
TOTALS
c + d
a + b
TOTALS a+b+c+d
b + d
a + c

RESULTS OF GOLD
STANDARD TEST
EXPERIMENTAL
TEST
TEST
POSITIVE
TEST
NEGATIVE
DISEASE
PRESENT
DISEASE NOT
PRESENT
TRUE (+)
a
FALSE (+)
b
FALSE (-)
c
TRUE (-)
d
TESTS ARE RARELY
100% ACCURATE
THEY MUST BE
COMPARED AGAINST
THE GOLD STANDARD
TOTALS
c + d
a + b
TOTALS a+b+c+d
b + d
a + c
SENSITIVITY
SPECIFICITY

SENSITIVITY AND SPECIFICITY
SENSITIVITY = PATIENT (+) TEST (+)
Probability that patient WITH disease
will have ABNORMAL result
(instead of False Negative)
SPECIFICITY = PATIENT (-) TEST (-)
Probability that patient WITHOUT disease
will have NORMAL result
(instead of False Positive)
OVERALL DISCRIMINATION OF TESTS
High SENSITIVITY = low number false negatives
High SPECIFICITY = low number of false positives
Best accuracy if both factors are close to 100%

SENSITIVITY = a / (a + c)
PATIENT (+) TEST (+)
SPECIFICITY = d / (b + d)
PATIENT (-) TEST (-)
POSITIVE PREDICTIVE VALUE = a / (a + b)
If pt tests (+), what is the likelihood s/he has the disease?
NEGATIVE PREDICTIVE VALUE = d / (c + d)
If pt tests (-), what is the likelihood s/he does NOT have the disease?
PREVALENCE = (a + c) / (a + b + c + d)
ACCURACY = (a + d) / (a + b + c + d)
Proportion of results that correctly identify pts with and without disease
(True + and True - as proportion of all results)
LIKELIHOOD RATIO = sensitivity / (1 - specificity)
How likely is it that + result accurately indicates disease, and - result no disease?

LIKELIHOOD RATIO (LR)
Because Sensitivity and Specificity are NOT always 100%
How likely is it that + result accurately indicates disease, and - result no disease?
LIKELIHOOD RATIO FOR A POSITIVE RESULT (LR+)
Probability of (+) result in diseased subject
divided by
Probability of (+) result in a healthy subject
- or worded differently -
Sensitivity
divided by
100% - Specificity
LIKELIHOOD RATIO FOR A NEGATIVE RESULT (LR-)
100% - Sensitivity
divided by
Specificity
DISCRIMINATION = ZERO IF LR = 1

EVALUATING STUDY RESULTS
Example:
Mortality rates in 4444 pts x 5.4 trial years:
11.5% Placebo
8.2% Medicine
RRR 29% (Relative Risk Reduction)
ARR 3.3% (Absolute Risk Reduction)
NNT 30 (Number needed to treat for 5.4
years to save 1 life

QALY = QUALITY ADJUSTED LIFE YEAR
QUALITY RATING
NOT specific for disease or treatment!
Value rating - subject to different values of patients, physician, community
• Patient-based
• Economy-based - cost-utility/cost-effectiveness analysis
Compares outcomes of conditions or intervention(s)
• State of health - vs -Time spent in it

TEST RESULTS: VARIABILITY
FACT: Study results may vary.
Group too small
Not representative
of larger group
Etc.
Age, sex, race, condition, culture, etc.
Compliancy issues (patient and
physician!)
May be discovered
or identified through
study

FACT: Variability may or may not be significant
Group too small
Not representative
of larger group
Etc.
physician!)
May be discovered
study

Group too small
Not representative
of larger group
Etc.
physician!)
May be discovered
study
Obviously faulty studies should be eliminated.

Group too small
Not representative
of larger group
Etc.
physician!)
May be discovered
study
Some variability should be expected in the rest.

Group too small
Not representative
of larger group
Etc.
physician!)
May be discovered
study
Some factors are completely unexpected.

and variability due to
The statistics allow us to distinguish between

PROBABILITY CHANCE
WARNING
PROBABILITY should
NOT
be confused with
CHANCE!
Statistically significant
Results are measurable
and predictable
Affected by sample size
(1 in 20 is less convincing than
1 in 10,000)
No statistical
significance
Random, unpredictable

A Study in Probability…
QUESTION #1:
What percentage of patients will develop diarrhea while
taking Antibiotic A?
QUESTION #2:
Will the results be the same, better or worse on
Antibiotic B?
PROBABILITY

50
50
50
50
50
50
50
50
50
50
50
50
50
50
50
50
50
50
50
50
20 study
groups
50 pts each
study
1000 patients
total
QUESTION #1:

4 6 10
8 12 14 16%
Conduct
studies
Organize
results
NUMBER
OF
STUDIES
PERCENTAGE OF PATIENTS WITH DIARRHEA
QUESTION #1:

4 6 10
8 12 14 16%
NUMBER
OF
STUDIES
IDENTIFY STATISTICALLY SIGNIFICANT RESULTS
The Bell Curve

Most COMMON result =
4 6 10
8 12 14 16%
MODE
NUMBER
OF
STUDIES

The CENTER of distribution =
4% 6 8 12 14 16%
MEDIAN
NUMBER
OF
STUDIES
10

MODE (most COMMON result) = 10%
MEDIAN (the CENTER of distribution) = 10%
NUMBER
OF
STUDIES
DETERMINE RESULTS OF STUDY
4 6 10
8 12 14 16%

“10% of patients will develop diarrhea while
taking Antibiotic A.”
NUMBER
OF
STUDIES
10
4 6 8 12 14 16%
CONCLUSION

4 6 8 12 14 16
NUMBER
OF
STUDIES
Wait… What about the other 15 study groups?
You can’t just
ignore them!
10
Or can
you?

4 6 8 12 14 16
You can’t just
ignore them!
10
Or can
you?
15 groups
x 50 per group
= 750 patients
(75%!)

4 6 8 12 14 16
You can’t just
ignore them!
10
Or can
you?
Studies must
account for
ALL patients

4 6 8 12 14 16
NUMBER
OF
STUDIES
You can’t just
ignore them!
10
Or can
you?
Results should
not be ignored,
but
STATISTICAL
SIGNIFICANCE
may be
questioned.

NUMBER
OF
STUDIES
4 6 8 12 14 16
10
My head is
starting to feel
heavy…
So, how is
STATISTICAL
SIGNIFICANCE
determined?

PROBABILITY is determined by it.
CHANCE is not related to it at all!
STATISTICAL SIGNIFICANCE
How is it determined?

Let’s use our study on “Antibiotic A” as the example
4 6 8 12 14 16%
10
NUMBER
OF
STUDIES 20 groups were tested.
Only 1 of 20 groups landed at each end of the Bell Curve….
WHY?
SAMPLE
VARIATION?
or CHANCE?

“There is a 1 in 20 chance that other patients
will land in these categories.”
But that
would NOT
be a correct
statement!
It is tempting to say,
4 6 8 12 14 16%
10
NUMBER
OF
STUDIES

“There is a 1 in 20 chance that other patients
will land in these categories.”
Why?
Because
CHANCE
can not be
used to
predict
future
results!
It is tempting to say,
4 6 8 12 14 16%
10
NUMBER
OF
STUDIES

CHANCE is based on RANDOM possibility.
Example: THE COIN TOSS
Coins tossed: 20
“Heads” 17 (85%)
“Tails” 3 (15%)
Statistical Significance: ZERO!
The next coin toss will still produce a random result!
Random results can not be used to calculate
Statistical Probability.

4% 6 8 12 14 16%
10
“There is a 1 in 20 chance that patients will
land in one of these two categories.”
So instead of measuring “chance”…
1 in 20
chance
1 in 20
chance

4% 6 8 12 14 16%
10
We need to determine the PROBABILITY!
1 in 20
chance
Translates into
5%
probability
“There is a 5% probability that patients will
land in one of these two categories.”

4% 6 8 12 14 16%
There is a
5% probability
that 4% of
patients will
develop
diarrhea on
Antibiotic A.
There is a
5% probability
that 16%of
patients will
develop
diarrhea on
Antibiotic A.
10
The results now look like this:

4% 6 8 12 14 16%
10
1 in 20
chance
Probability
= 5%
FRACTION PERCENTAGE
And now… Let’s abbreviate it some more!
p = 0.05
DECIMEL

4% 6 8 12 14 16%
10
1 in 20
chance
FRACTION PERCENTAGE
p = 0.05
DECIMEL
…by changing “% probability” to the “p value”
Probability
= 5%

4% 6 8 12 14 16%
10
1 in 20
chance
p = 0.05
FRACTION
DECIMEL
PERCENTAGE
The “p value” is statistically important!
Probability
= 5%

4% 6 8 12 14 16%
10
1 in 20
“chance”
p = 0.05
It determines statistical PROBABILITY.
“p value”
Probability
= 5%

4% 6 8 12 14 16%
10
PROBABILITY vs CHANCE
…but PROBABILITY is
very important!
It tells us the likelihood that
something will happen.
So, CHANCE
has ZERO
significance
NUMBER
OF
STUDIES

“There is a 5% probability that our study patients
will fall into either of these categories.”
OUR PROBABILITY STATEMENT
p = 0.05 p = 0.05
4 6 10
8 12 14 16%

Anything less than 5% (p= 0.05)
MAY be due to chance.
THAT IS A LOW PROBABILITY!
p = 0.05 p = 0.05
4 6 10
8 12 14 16%

4 6 10
8 12 14 16%
p = 0.01
p = 0.01
And anything less than 1% (p= 0.01)
is MOST LIKELY due to chance!

Anything less than 5% (p= 0.05) MAY be due to chance.
4 6 10
8 12 14 16%
p = 0.01
p = 0.01
Anything less than 1% (p= 0.01) is MOST LIKELY due to chance.
p = 0.05 p = 0.05

(p= 0.05) becomes VERY SIGNIFICANT
But when compared to another study…
4 6 10
8 12 14 16
And (p= 0.01) becomes HIGHLY SIGNIFICANT!
18%

4% 6 10
8 12 14 16
Antibiotic A
NUMBER
OF
STUDIES
18%

4% 6 10
8 12 14 16
Antibiotic B
NUMBER
OF
STUDIES
18%

(p= 0.05) becomes VERY SIGNIFICANT
DIFFERENCE = STATISTICALLY SIGNIFICANT
4 6 10
8 12 14 16
And (p= 0.01) becomes HIGHLY SIGNIFICANT!
18%

The P value
Measures Probability
How often is this finding expected to occur?
Determines Statistical Significance
What is the likelihood these findings are TRUE or FALSE?
Do the comparative findings show a significant difference?
Meaningful ranges
p >0.05 Not significant
p <0.05 Statistically SIGNIFICANT
p <0.01 HIGHLY SIGNIFICANT!
Does probability provide PROOF?
NO! We could be misled by it.
The sample size is very important when determining probability!

SAMPLE SIZE and PROBABILITY
EXAMPLE:
100 pieces of fruit are in a bin: APPLES and ORANGES
You close your eyes and pick 10 of them:
Question: Does your sample accurately
represent what is in the bin?

Answer: No!
Larger samples provide a closer approximation of the
populations they represent... But the only way to get
100% proof is to examine “all of the fruit in the bin!”

The P value
Meaningful ranges
p >0.05 Not significant
p <0.05 Statistically SIGNIFICANT*
p <0.01 HIGHLY SIGNIFICANT!**
*Significance only means that CHANCE is an unlikely explanation for the
results
LIMITATION
The p value determines LIKELIHOOD… Not proof!
CAUTION
Statistical significance does not necessarily imply any clinical
significance!
EXAMPLE: Looking through a pinhole will improve vision in most people… But would
this be an appropriate treatment for your myopic patients? (Key Topics in EBM )

PATIENT
PHYSICIAN INFORMATION
Question
or
Problem
(The Three Major Components of EBM)

PATIENT
“A METHODOLOGICAL
MINEFIELD”

PATIENT
“A
METHODOLOGICAL
MINEFIELD”
INFORMATION
Difficult time
understanding
background
information
PHYSICIAN
Personal
priorities may
conflict with
yours

PATIENT
Recognize:
Needs
Choices
Preferences
Values
Socioeconomic
concerns
INFORMATION
Help the patient to
understand and
interpret available
information
PHYSICIAN
Respect the
personal
priorities of the
patient
Help the patient
to negotiate a
decision on
intervention,
treatment
CONFLICT?
SEPARATE THE ISSUES!

PATIENT
INFORMATION
PHYSICIAN
PATIENT
And then help the
patient pull it all
together again

KEY POINTS
PARADIGM SHIFT
OLD: Doctor had authority
(despite the pile of unread journals!)
NEW: Current Best Evidence leads medical practice
but it MUST be individually applied
THE INDIVIDUAL PATIENT
Every patient is different. Treat YOURS and not others
The “ideal” course of action is not necessarily best for THIS patient.
EBM + Psychosocial factors =
THIS patient should be advised to take
THIS therapy at
THIS point in time.

INFORMATION
Adequate resources?
Ease or Difficulty of finding and getting desired information?
Costs?
INTERVENTION
Patient response or acceptance?
Ease or Difficulty of Application?
Clinical outcomes?
EBM PROCESS
EFFECT ON PRACTICE
Will this particular experience change our thinking or practice?
SELF EVALUATION
How did we do? (Question, Search, Appraise, Apply)
How could we improve our own EBM performance?
Evaluate

EBM: PROS, CONS and LIMITATIONS

PROS
Clinicians update knowledge base routinely
Improved understanding of research methods
Physician becomes more critical in use of data
Increased confidence in management decisions
Increased computer literacy, data search technology
Better reading habits
Provides framework for group problem solving, team generated
practice

Transforms weakness or paucity of knowledge into positive change
OK to be uncertain
OK to be skeptical
OK to be flexible
Integrates medical education, research and clinical expertise
Can be learned by non-clinicians – other HCWs, patient groups,
purchasers, etc.
Allows us to keep up with our better-educated patients!

Increased contribution of junior MDs
Increased patient benefit
Better communication with patients re: rationale of management
decisions
Promotes better and more appropriate use of limited resources
May reduce costs or medical care or practice by eliminating outdated or
unnecessary factors
Can be learned at any stage of physician’s career

CONS
Time consuming
Information overload
Time to learn and practice
Time may be needed for team conferencing, planning and review
Takes $$$ to establish resource infrastructure – library, office, etc.
computers, peripherals

Internet costs
Programs, software information, CD-ROMS
Subscription costs – online and paper resources
May increase cost of care (but hopefully offset by elimination of
unnecessary medical interventions, tests, journals, etc. – plus save time
in getting proper intervention)
Online references made to unavailable journals or references
Exposes gaps in the evidence (but provides ideas for researchers!)

Requires computer skills (but can be done with minimal
computer literacy and skill)
May expose your current practice as obsolete or dangerous
(loss of authority and respect)

LIMITATIONS
Lack of evidence (shortage of studies)
Difficulty applying evidence to care of a particular patient
Barriers to the practice of high quality medicine
Lack of skills (search, appraise, etc.) (foster development of new
skills!)
Lack of time to learn and practice EBM (promotes lifelong learning thru
better focus)
Lack of physician resources for instant access to evidence (EBM has
worldwide applicability)

RESTRICTED AVAILABILITY OF LAB TESTS
NON-TEXTBOOK CASE
co morbidity, additional risk factors
AFFORDABILITY (MD & PT)“I can’t afford to practice EBM.”
Language barriers – available evidence may be unreadable, should be
included

Physician attitude: Can be the greatest limitation!
“It decreases the importance of my clinical expertise”
(that’s a necessary component!)
“It only applies to those involved in research.”
(promotes cooperation among multiple physicians)
“It ignores patient values and preferences.”
“It’s just another cookbook approach to medicine.”
“It’s a poorly disguised way to cut medical costs.”
(cost of care may actually increase)
“It’s a way to ration care and resources.”
(Provides better utilization of avail resources)

DISAGREEMENT
Pt’s comfort, choice, acceptance, values preferences
Vs MD’s recommendations
DOES RISK OR SIDE EFFECTS OF TREATMENT OUTWEIGHT THE
BENEFITS?

The unanswered question…
“DOES EBM REALLY MAKE A DIFFERENCE?”
Effect of practicing EBM on patient outcome is actually
unknown – no studies done
EBM good based on population studies:
(ie: Pts who rec’d ___ generally fare better than those who don’t)

EBM IN DEVELOPING COUNTRIES
LIMITED RESOURCES
May help to eliminate unnecessary or poor quality
screening tests (ie: resting EKG to screen for
CAD = high false negative and false positive
rates)
LIMITED DRUG REGULATION
Approval for drug marketing easy - promotes
insurgence of new drugs for questionable
indications, limited effectiveness, false claims,
inflated prices based on ad response (include
“more expensive is better”)

LIMITED CAPACITY FOR CME
Drug companies - may sponsor meetings that are little
more than captive marketing sessions or biased
education sessions (drug education vs promo)
Result may be push for more expensive, less effective
treatments (ie push for CCB’s over BB’s) - calc channel
blockers over Beta Blockers

LIMITED ACCESS TO LITERATURE DATABASES
Desktop computer with CD ROM reader and modem
($900)
Electricity
1 yr subscription to MedLine on CD ROM (?500)
Internet connection $25/mt
Convince administrators of expense:
Publicly cite how searches help with lectures, research
and patient care management decisions
Get equipment from drug companies
(usually strings attached)

LIMITED ACCESS TO ADEQUATE LIBRARY FACIILITIES
ALMOST INEVITABLE IN DEVELOPING COUNTRIES
Identify resources via search, but then unable to retrieve articles!
A top EBM practitioner (Philippines) recommends:
1. Top 3 medical libraries in your country
2. Multinational drug company libraries
3. Friends and colleagues - including in other countries

QUESTIONABLE APPLICABILITY OF ARTICLES
RETRIEVED
Article describes a treatment that worked in one country, but
seems impossible in yours
Check…
• Are there pathophysiologic differences?
• Will patient differences diminish the treatment response?
• Patient compliance issues?
• Provider compliance issues?
• Co-morbid conditions which will alter the benefits or risks?

OBSTACLES TO TEACHING OR LEARNING EBM
Your Hospital or Institution does not reimburse for time spent on
Continuing Medical Education programs
The standard 5-day workshop would be far too costly to provide or
attend!
Need to learn the basics - computer skills, etc.
TRY THESE!
Combine efforts to learn more and practice EBM with handful of
colleagues (small group learning)
Ask about basis for information provided by drug reps, medical supply
companies, etc. It will prompt them to provide you with on the spot
teaching and better information, too!

EBM LIBRARY
BASIC REQUIREMENTS
Convenient – easy access at point of contact with patient if
possible
Current – Up to date information
Electronic Database – Should be included
• Online
• CD-ROM

ELECTRONIC DATABASES
Evidence-Based Medicine Reviews (EBMR) – from Ovid
(ovid.com)
- combines Cochrane, Best evidence, Evidence Based
Mental health, EB Nursing, Cancerlit, healthstar, AIDSline,
Medline, and journal links (Described by one EBM specialist
as “the best”)
Cochrane Library – “Gold Standard” for systematic reviews
Best Evidence
Medline – world’s largest, free resource – over 10 million
references

PERSONNEL
Medical Librarian
Informatics Specialist
“We can learn a great deal about current best information
sources from librarians and other experts in medical
informatics, and should seek hands-on training from them
as an essential part of our clinical training.”
(ch 2 p29-30 – Blue circled 2)

PRINTED RESOURCES
TEXTBOOKS
most obsolete!
Some updated yearly, plus heavy references and scientific
evidence for support
Clinical Evidence (BMJ Publishing Group & ACP – 1999-
present)
Evidence-Based On Call (http//cebm.jr.ox.uk/eboc/eboc.html)
Up To Date (General medicine, CD format, Medline abstracts
used for evidence)
Scientific American Medicine – limited references from
Medline, Harrisons

JOURNALS
Traditional Journals
subject to author submissions
specialists need to read and evaluate
may subscribe to services that send articles of interest to
your specialty
timely, instant information at time of publication
Ex: NEJM, Clinical Nephrology, etc.
Evidence Based journals
selects best studies from multiple journals of interest,
summarizes best evidence
Good for use by generalists
Lag time from original publication: 3-6 months
Ex: Evidence Based Medicine, Evidence Based
Nursing, Evidence Based CV Medicine, etc.

SPECIAL RESOURCES
WHO Blue trunk
Hinari
PATIENT RESOURCES
Medical treatments www.nlm.nih.gov/medlineplus
Medical guidelines www.guideline.gov

THE NEXT LEVEL: ADVANCED EBM
SUMMARIZE AND STORE INFORMATION
Future reference
SHARING INFO
Local Colleagues
Author paper
TEACHING
New skills or treatments
EBM practices
OTHER APPLICATIONS
Care of the individual patient
Team protocols
Hospital or practice guidelines

EBM in Medical Education
Message to medical educators from Trisha Greenhalgh, MD,
co-author of Evidence Based Health Care Workbook:
“An important challenge for medical educators… is to
recognize that the competent student (and clinician!) is one
who knows how to cope with an immense and rapidly
changing body of knowledge and not one who excels in
recalling the traditional or memorizing the ephemeral.
The deans of medical and nursing schools must develop an
infrastructure that allows problem-based, self-directed
learning methods to develop within the didactic, lecture-
based curricula, which have seen no fundamental changes
for two centuries or more.”

ADVANCED EBM: ADVANCED APPLICATIONS
APPLY METHODS TO…
Care of the individual patient
Team protocols
Hospital or practice guidelines
Continued Learning: problem-based approach
Teaching

SELF-DIRECTED LEARNING
JAMA Series of User Guides
“Clinical Epidemiology: A Basic Science for Clinical
Medicine”
Week-long workshops
On-the-job learning (in your own practice)

Developed and presented by
Judy Tarselli, RN
Dubai, UAE
Karachi, Pakistan
October 2003
Organized by NKF cyberNephrology
University of Alberta, Canada
www.cyberNephrology.org
Special thanks to our sponsors at Janssen-Cilag of Dubai

Ebm presentation

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Ebm presentation

Similar to Ebm presentation (20)

Recently uploaded

Recently uploaded (20)

Ebm presentation