Ebm & critical appraisal

EVIDENCE BASED MEDICINE
&
PRINCIPLES OF CRITICAL
APPRAISAL
1st year Residents
Dept. Of Psychiatry
MGMCRI

EBM & CRITICAL APPRAISAL
1st year Residents ● Department of Psychiatry ● MGMCRI ● Pondicherry
● EBM (Evidence-based medicine) is a systematic
way of obtaining clinically important information
about aetiology, diagnosis, prognosis and treatment.
● EBM can be applied to variety of medical
procedures.
● In psychiatry, useful to assess the value of
Therapeutic interventions.
Ref: SOTP
2
Evidence Based Medicine

● Asking answerable questions
● Search for the evidence
● Critical appraisal of results
● Action to take from your findings
● Evaluate your new or amended practice
Ref: UCL 2011
 Five steps of EBM:
3
Evidence Based Medicine

● Critical appraisal is the process of carefully and
systematically examining research to judge its
trustworthiness, value and relevance in a particular
context.
● Critical appraisal is an important element of
evidence-based medicine.
Ref : Burls 2009
 Definition
4
Critical appraisal

● Combat information overload
● Identify papers that are clinically relevant
● Continuing Professional Development (CPD) –
critical appraisal is a requirement for the evidence
based medicine component of many membership
exams
Ref: UCL 2011
 Essentials
5
Critical appraisal

● The Critical Appraisal Skills Programme (CASP)
aims to help people develop the necessary skills to
make sense of scientific evidence.
● It has produced appraisal checklists covering:
 Validity
 Results
 Relevance
Ref: Burls 2009
6
Critical Appraisal Skills Programme (CASP)

● Assessed by looking at the methodologies of the
study in more detail (not just jumping to the
conclusions)
● Are the methods robust?
● Did the study address a clearly focused issue?
● How were the participants of the study assigned?
Ref: NHS 2011
7
Validity

● Look at the results in more detail – are they credible
and repeatable?
● How was the data collected?
● What are the key findings?
● How significant are the results?
Ref: NHS 2011
8
Reliability

● Can the results be applied to my local health
community?
● Were all important outcomes considered?
● Do you need further evidence before finding an
answer to your question?
Ref: NHS 2011
9
Relevance

● Is the study design suited to fulfill the aims of the
study?
● Is it stated whether the study is confirmatory,
exploratory or descriptive in nature?
● What type of study was chosen, and does it permit
the aims of the study to be
● addressed?
● Is the study's endpoint precisely defined?
● What statistical measure is employed to
characterize the endpoint?
10
Questions on methodology

Do epidemiological studies, for instance, give the
incidence (rate of new cases), prevalence (current
number of cases), mortality (proportion of the
population that dies of the disease concerned),
lethality (proportion of those with the disease who die
of it) or the hospital admission rate (proportion of the
population admitted to hospital because of the
disease)?
● Are the geographical area, the population, the study
period (including duration of follow-up), and the
intervals between investigations described in detail?
Ref: Du Prel 2009
11
Questions on methodology

● Does the study pose scientifically interesting
questions?
● Are statements and numerical data supported by
literature citations?
● Is the topic of the study medically relevant?
● Is the study innovative?
● Does the study investigate the predefined study
goals?
● Is the study design apt to address the aims and/or
hypotheses?
12
Critical questions

● Did practical difficulties (e.g. in recruitment or loss to
follow-up) lead to major compromises in study
implementation compared with the study protocol?
● Was the number of missing values too large to
permit meaningful analysis?
● Was the number of cases too small and thus the
statistical power of the study too low?
● Was the course of the study poorly or inadequately
monitored (missing values, confounding, time
infringements)?
13
Critical questions

● Do the data support the authors' conclusions?
● Do the authors and/or the sponsor of the study
have irreconcilable financial or ideological conflicts
of interest?
Ref: Du Prel 2009
14
Critical questions

● Document
● Passive observation
● Participant observation
● In depth interview
● Focus group
Ref: UCL 2011
15
Qualitative studies

● Case report
● Case series
● Case control study
● Cohort study
● Randomized Controlled Trial (RCT)
● Systematic review
● Meta-analysis
Ref: UCL 2011
16
Quantitative studies

1st year Residents ● Department of Psychiatry ● MGMCRI ● Pondicherry 17
Different research questions require
different study designs
Purpose of investigation Study type
Investigation of rare diseases Case-control studies
Investigation of exposure to rare
environmental factors
Cohort study in an exposed
population
Investigation of exposure to multiple
agents
Case-control studies
Investigation of multiple endpoints Cohort studies
Estimation of incidence in exposed
populations
Exclusively cohort studies
Investigation of cofactors that vary
over time
Preferably cohort studies
Investigation of cause and effect Intervention studies
Ref: Du Prel 2009

1st year Residents ● Department of Psychiatry ● MGMCRI ● Pondicherry 18
Hierarchy of evidence
Ref : Haynes 2006

● RCT - most appropriate research design for studying
effectiveness of a specific intervention or treatment.
● Participants are randomly assigned to 2 (or more)
groups: one (or more) experimental group(s)
receiving the intervention, and a control group
receiving a placebo.
● The groups are then followed up to see what
differences result.
● It is gold standard in testing the efficacy of an
intervention (therapy/prevention).
Ref: Hill 2001
19
RCT

Advantages
– Allow for rigorous evaluation of a single
variable.
– Potentially eradicate bias.
– Allow for meta-analysis.
Ref: UCL 2011
 Advantages & Disadvantages:
20
RCT

Disadvantages:
– Expensive.
– Time consuming.
– Ethically problematic at times-a trial is
sometimes stopped early if dramatic effects are
seen.
Ref: UCL 2011
 Advantages & Disadvantages:
21
RCT

• Allocation (randomisation, stratification,
confounders)
• Blinding
• Follow up of participants(intention to treat)
• Data collection(bias)
• Sample size
• Presentation of results (clear and precise)
• Applicability to local population.
Ref: UCL 2011
 Factors to look for:
22
RCT

● Systematic review – identifies and critically
appraises all research on a specific topic.
● It Combines valid studies.
● Increasingly important in evidence based medicine.
● It is different from review article (which is a summary
of more than one paper on a specific topic and
which may or may not be comprehensive)
Ref: UCL 2011
23
Systematic review

● Advantages:
Allow for rigorous pooling of results
May increase overall confidence from small
studies
Potentially eradicate bias
May be updated if new evidence becomes
available
May have the final say on a clinical query
May identify areas where more research is
needed
Ref: UCL 2011
 Advantages & Disadvantages
24
Systematic review

● Disadvantages:
Expensive
Time consuming
May be affected by publication bias-a test called
Funnel Plot can be used to test for publication
bias
Normally summarise evidence up to two years
before(due to the time required for the execution
of the systematic review).
Ref: UCL 2011
 Advantages & Disadvantages
25
Systematic review

• Literature search (did it include published and
unpublished materials as well as non-English
language studies? Was personal contact with
experts sought?)
• Quality-control of studies included (type of
study; scoring system used to rate studies;
analysis performed by atleast two experts)
• Homogeneity of studies
• Presentation of results(clear,precise)
• Applicability to local population
Ref :
 Factors to look for:
26
Systematic review

● Meta-analysis- a systematic review that uses
quantitative methods to summarise the results.
● Four steps of meta-analysis:
– Identifying studies
– Determining eligibility of studies
• Inclusion: which ones to keep
• Exclusion: which ones to throw out
– Abstracting data from the studies
– Analyzing data in the studies statistically
Ref : UCL 2011
27
Meta-analysis

● It is a systematic and objective way of assessing the
usefulness of published research
● The route to closing the gap between research and
practice
● It is a common sense approach to reading and user
friendly tools are available to help anyone develop
these skills.
● Keep up to date for your own professional
development
Ref :Hill 2001
 Pros
28
Critical appraisal

● It can be time-consuming initially
● It can be dispiriting if it highlights a lack of good
evidence
● It does not always provide the reader with the ‘easy’
answer or the answer one might have hoped for. It
may highlight that a favoured intervention is in fact
ineffective.
Ref : Hill 2001
 Cons
29
Critical appraisal

● The systematic deviation of the results of a study
from the truth because of the way it has been
conducted, analysed or reported – Bias
● No study is perfect and free from bias
● Systematically check that the researchers have
done all they can to minimise bias
● Study which is sufficiently free from bias is said to
have Internal validity.
Ref : UCL 2011
30
Bias

 Types of Bias
31
Bias
Types Definition
Selection bias Biased allocation to
comparison groups
Performance bias Unequal provision of care
apart from treatment under
evaluation
Detection bias Biased assessment of
outcome
Attrition bias Biased occurrence and
handling of deviations from
protocol and loss to follow up
Ref: Burls 2009

● Randomisation- participants are randomly allocated
to treatment or control group.
● Acceptable methods- random numbers, either from
tables or computer generated.
● Unacceptable methods- last digit of date of birth,
date seen in clinic, etc
● Stratified randomisation- used to avoid confounding
factors, i.e. to ensure equal distribution of
participants with a characteristic thought to affect
prognosis or response.
Ref : UCL 2011
 Randomisation & blinding
32
Mechanism to control bias in RCT

● Blinding- masking who is getting treatment and
control.
● Single blinding: participants do not know.
● Double blinding: neither the participants nor those
giving the intervention know.
● Triple blinding: statisticians doing the analysis also
do not know.
Ref : UCL 2011
 Randomisation & Blinding
33
Mechanism to control bias

● Systematic reviews avoid bias by:
Containing a statement of objectives, materials and
methods
Following an explicit and reproducible methodology
Ref : UCL 2011
34
Mechanism to control bias in Systematic
reviews

● CONSORT (Consolidated Standards of Reporting
Trials) guidelines for randomized trials.
● QUORUM (Quality of Reporting of Meta-analyses)
● MOOSE (Meta-analysis Of Observational Studies in
Epidemiology) for meta-analyses
● STROBE(Strengthening the Reporting of
Observational Studies in Epidemiology) for various
sorts of observational studies.
● PRISMA (Preferred Reporting Items for Systematic
Reviews and Meta-Analyses).
35
Standard guidelines

Ebm & critical appraisal

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to Ebm & critical appraisal

Similar to Ebm & critical appraisal (20)

Recently uploaded

Recently uploaded (20)