high information

1. Addis Ababa university college of health science department of
nursing and midwifery post graduate program
Course Title: Evidence based practice and clinical Audit
Assignment :Critical appraisal of evidence with example
1. Mulugeta Abeneh Mulu…………..GSR/4375/16
2. Workalem Tilahun …….…………GSR/1552/16
3. Simeneh Tsegaye ………………...GSR/1854/16
4. Dersolign Berihun………………...GSR/3503/16
Submitted to Instructor Tefera M
(Ass’t. Professor)
January 2024
ETHIOPIA (A.A)

2. ACKNOWLEDGEMENTS
First of all, we would like to thank almightily
God’s for his kindness, love and helping us for
everything that couldn‘t be done beyond him.
Next, we gratefully acknowledge to our dear
instructor :Tefera M (Ass’t. Professor) for his help
to build us with plenty of knowledge on the area of
interest.

3. OBJECTIVE
• Develop an understanding of the meaning of critical
appraisal of evidence
• Describe how level of evidence are used in critical
appraisal of evidence
• Identify critical appraisal cheek list
• Examine piece of evidence with critical appraisal check
list
• Identify PICOT type question for best available best
evidence
• Introduction to the clinical problem that we will address
the intervention

4. Evidence based practice
What is EBP?
• Evidence based practice
(EBP) is 'the integration of
best research evidence
with best available
scientific research, clinical
expertise and patient
values, which when applied
by practitioners will
ultimately lead to improved
patient outcome
Internal & external evidence

5. Critical appraisal of evidence
Defined
1. Assessing the strength of
the scientific evidence and
2. Evaluating the research for
its quality and applicability
in health care decision
making
 Core skill needed to use
evidence to support clinical
decision
One of the step of EBP

6. Cont.'s…
Strength of evidence :Grading of strength of
evidence should incorporate:-
I. Quality ;- the extent to which bias was
minimized (internal validity)
II. Quantity :- the extent of the magnitude of effect
,number of study and sample size or power
III. Consistency :- the extent to which similar and
different study design report similar findings

7. Cont.'s…
Strength of evidence
 Evidence exist on the
continuum of rigor
 Amount of research
attention or maturity of
science varies, thus
evidence varies
 Type of research design
reflects the strength of the
evidence, which is known as
level of evidence
Level of evidence
• Ranking as to how well the
evidence informs clinical
intervention
• Experts have developed a
number of taxonomies to rate
the strength of evidence
• The stronger the level of
evidence the greater
confidence that the evidence
worth to practice
• Level of evidence are based
on research design

8. Level of evidence
Rating system for the hierarchy of
evidence
• Level I:Evidence from a SR or meta
–analysis of all RCT
• Level II: Evidence obtained from at
least one well –designed RCT
• Level III: Evidence obtained from
well design quasi-experiment
• Level IV : Evidence from case
control and cohort studies
• Level V: Evidence from systematic
review (SR) of qualitative and
descriptive study
• Level VI: Evidence from single
descriptive or qualitative studies
• Level VII: Evidence from expert
opinion

9. General critical appraisal overviews to
all studies
 Given that all research is not perfect, users of research must
learn to carefully evaluate research reports to determine
their worth to practice through critical appraisal.
 The critical appraisal process hinges on three overarching
questions that apply to any study
• Are the results of the study valid? (Validity)
• What are the results? (Reliability)
• Will the results help me in caring for my patients?
(Applicability)
 This provides clinicians to interpret the quality of studies
and determine the applicability of the synthesis of multiple
studies’ results to their particular patients.

10. Are the Study Results Valid? (Validity)
• Validity refers to the accuracy of the results of the
study were obtained via sound scientific methods
• Bias and/or confounding variables may compromise
the validity of the findings.
• we need to consider how study participants were
chosen.
• Were they selected at random or by some other
method?
• Were measures taken to decrease biased results based
on how participants were selected?
• we need to consider the accuracy and completeness of
the data.

11. Cont.’s…
• Were the study used randomization to reduce
bias and confounding variable
• To minimize this bias, participants and those
evaluating outcomes of the study are kept
blind or “in the dark” about who receives
which intervention.
• Followed up were the study should begin and
end with the same number of patients in each
group.

12. What Are the Results? (Reliability)
Quantitative studies use statistics to report their findings.
• To use the results of quantitative studies need a
general understanding of how to interpret the
numerical results
• The main concerns are the size of the intervention’s
effect (the effect size) and how precisely that effect
was estimated.
• The effect is the rate of occurrence in each of the
groups for the outcome of interest
• The study’s results to evaluating how likely it is that the
intervention will have the same effect when clinicians
use it in their practices

13. CONT,S’…
Reliability
• This part of critical appraisal examines the numerical data reported in the
results section of a study
• The total number of participants approached and the number consenting to
participate in the study should be reported (i.e. to follow-up)
• Statistical tests should conducted that to determine if the effects differ
significantly between groups
• Strength association were conducted to present the treatment effect through
(OR,RR,NNT etc.)
• When appraising a study, clinicians (and patients) want to know the
importance of these results for the clinical decision ,this is referred to as
clinical significance
• How Precision in the measurement of effect(statistical significance and
confidence interval
• The narrow the CI the greater chance for the result reliability

14. Will the Results Help Me in Caring for
My Patients? (Applicability)
• This question is also known as clinical significance
• Clinicians who are appraising evidence should always
keep application to patients in mind as the ultimate goal
• we need to assess whether your patient or patient
population is similar to those in the study
• What are the risks and benefits of treatment?
• Is the treatment feasible in my clinical setting?
• What are my patient’s values and expectations for the
outcome?

15. Rapid critical appraisal checklists for
specific study designs
 Systematic review :-is a complex piece of research that aims to identify
,select and synthesis all research published on a particular question or
topic
• Provide state of the science conclusion bout evidence supporting benefit
and risk of a given health care practice.
• Most powerful and useful evidence available
• Refers to summary that uses a rigorous scientific approach to combine
result from a body of original research studies into a clinically meaning
whole.
 Meta-analysis
• Statistical approach to synthesizing the result of a number of studies –
summarize result of all studies included in the review
• Produce a larger sample size and thus greater power to determine the true
magnitude of an effect ,yielded a summary statistic

16. Rapid Critical Appraisal of Systematic Reviews and
meta analysis of Clinical
Interventions/Treatments
1. Are the results of the review valid?
a) Are the studies contained in the review randomized
controlled trials? Yes/ No
b) Does the review include a detailed description of the
search strategy to find all relevant studies? Yes /No
c) Does the review describe how validity of the individual
studies was assessed (e.g., methodological quality,
including the use of random assignment to study groups
and complete follow-up of the subjects)? Yes/ No
d) Were the results consistent across studies? Yes /No
e) Were individual patient data or aggregate data used in the
analysis? Individual /Aggregate

17. Cont.'s’…
2. What were the results?
a) How large is the intervention or treatment
effect (OR, RR, effect size, level of
significance)? _________________
b) b. How precise is the intervention or
treatment (CI)? _________________

18. Cont.’s’
3. Will the results assist me in caring for my patients?
a. Are my patients similar to the ones included in the review? Yes/ No
b. Is it feasible to implement the findings in my practice setting? Yes
/No
c. Were all clinically important outcomes considered, including risks
and benefits of the treatment? Yes/ No
d. What is my clinical assessment of the patient and are there any
contraindications or circumstances that would inhibit me from
implementing the treatment? Yes/ No
e. What are my patient’s and his or her family’s preferences and values
about the treatment that is under consideration? Yes/ No
© 2005 Fineout-Overholt & Melnyk. This form may be used for
educational, practice change & research purposes without permission.

19. Cont.…
Randomized controlled trial(RCT)
• Experimental studies are the gold standard of
research design(randomization of participant to
treatment and control group, rigorous method
used to minimize bias)
• Provide most valid, dependable research
conclusion about clinical effectiveness of an
intervention and establishing cause and effect
• Allow as to say with a high degree of certainty
that the intervention we used was the cause of the
outcome

20. Rapid Critical Appraisal Checklist for a
Randomized
Clinical Trial (RCT)
1. Are the results of the study valid?
a. Were the subjects randomly assigned to the experimental and control groups ? Yes
/No /Unknown
b. Was random assignment concealed from the individuals who were first enrolling
subjects into the study? Yes /No Unknown
c. Were the subjects and providers blind to the study group? Yes /No /Unknown
d. Were reasons given to explain why subjects did not complete the study? Yes/ No/
Unknown
e. Were the follow-up assessments conducted long enough to fully study the effects of
the intervention? Yes/ No/ Unknown
f. Were the subjects analyzed in the group to which they were randomly assigned? Yes/
No/ Unknown
g. Was the control group appropriate? Yes/ No/ Unknown
h. Were the instruments used to measure the outcomes valid and reliable? Yes /No
/Unknown
i. Were the subjects in each of the groups similar on demographic and baseline clinical
variables? Yes/ No/ Unknown

21. Cont.….
2. What are the results?
a. How large is the intervention or treatment
effect (NNT, NNH,effect size, level of
significance)? ____________________
b. How precise is the intervention or treatment
(CI)? ____________________

22. Cont.…..
3. Will the results help me in caring for my patients?
a. Were all clinically important outcomes measured? Yes /No
/Unknown
b. What are the risks and benefits of the treatment?
____________________
c. Is the treatment feasible in my clinical setting? Yes/ No/ Unknown
d. What are my patient’s/family’s values and expectations for
the outcome that is trying to be prevented and the treatment
itself? ____________________
© 2005 Fineout-Overholt & Melnyk. This form may be used for
educational, practice change & research purposes
without permission.

23. Rapid Critical Appraisal Checklist for
Descriptive Studies
Cohort –participant are studied over time,
study population share common characteristics
Case control-studies are address question
about harm or causation, investigate why some
people develop disease or behave the way they
do versus other who do not
Descriptive –the main objective is to describe
some phenomenon

24. Cont.
1. Are the results of the study valid?
• Were study/survey methods appropriate for the question? Yes /No
• Was sampling methods appropriate for the research question? Yes/ No
• Was sample size implications on study results discussed? Yes/ No
• Were variables studied appropriate for the question? Yes /No
Dependent variables are: Independent (outcome) variables are:
• Were outcomes appropriate for the question? Yes/ No
• Were valid and reliable instruments used to measure outcomes? Yes/ No
• Were the chosen measures appropriate for study outcomes? Yes/ No
• Were outcomes clearly described? Yes/ No
• Did investigators and/or funding agencies declare freedom from conflict of
interest? Yes/ No

25. Cont.
2. What are the results?
• What were the main results of the study?
• Was there statistical significance? Explain.
• Was there clinical significance? Explain.
• Were safety concerns including adverse events
and risk/benefit described? Yes/ No

26. Cont.
3. APPLICABILITY
Will the results help me in caring for my
patients?
• Are the results applicable to my patient
population? Yes/ No
• Will my patients’ and families’ values and
beliefs be supported by the knowledge gained
from the study? Yes/ No

27. Rapid Critical Appraisal of Evidence-Based Practice
(EBP)
Implementation or Quality Improvement (QI) Projects
……

28. Rapid Critical Appraisal of Qualitative
Evidence
………

29. Rapid Critical Appraisal Questions for
Cohort Studies
• ………

30. EVALUATION AND SYNTHESIS: FINAL STEPS IN
CRITICAL APPRAISAL
• Evaluation. The goal of evaluation is to determine how
studies within the body of evidence agree or disagree
by identifying common patterns of information across
studies
• A useful tool to help clinicians accomplish this is an
evaluation table. This table serves two purposes:
• First, it enables clinicians to extract data from the
studies and place the information in one table for easy
comparison with other studies; and
• Second, it eliminates the need for further searching
through piles of periodicals for the information.

31. Cont.
• Synthesis. In the synthesis phase, clinicians pull out key
information from the evaluation table to produce a snapshot of the
body of evidence.
• A table also is used here to feature what is known and help all those
viewing the synthesis table to come to the same conclusion.
• Synthesis occurs as clinicians enter the study data into the
evaluation table
• Each study is compared to the others for how it agrees or disagrees
with the others in the table
• Evaluation and synthesis tables differ in that evaluation tables
contain information about each individual study, while synthesis
tables contain only those aspects of the individual studies that are
common or unique across studies.
• We will see the difference with example

32. Example to critical appraisal evidence
•

33. NEC in VLBW infants
as sprit of inquire
 Necrotizing enter colitis (NEC) is a devastating condition
of the neonatal period characterized by bowel necrosis and
multisystem organ failure.
 It is well known that NEC is associated with prematurity
and particularly with extremely low birth weight .
 Necrotizing enter colitis is rare in term infants , in whom it
is usually associated with congenital anomalies, sepsis, or
hypotension .
 The morbidity and mortality are high, and optimal strategies
for treatment remain elusive, despite decades of research.

34. Cont.
 Epidemiology
• Necrotizing enter colitis (NEC) is affecting about 5% of all very
preterm or very low birth weight infants (VLBW: <1500 g) and
about 10% of all extremely preterm or extremely low birth
weight (ELBW: <1000 g) infants .
• The rate of NEC-associated acute mortality is generally reported
to be greater than 10% overall and more than 25% for infants
with NEC severe enough to require a surgical intervention.
• Infants with NEC have a higher incidence of nosocomial
infections and lower levels of nutrient intake, grow more slowly,
and have longer durations of intensive care and hospital stay.

35. Interventional type PICOT question
In very low birth weight infants (p),How
probiotic administration(I) compared to
placebo(C) reduce the incidence of NEC ? (O)
• Patient/population=VLBW infants
• Intervention=probiotic administration
• comparison= placebo
• Outcome= to reduce NEC
• Time ----

36. Search best available evidence
 How to search for the evidence to answer their clinical question
(shown here in PICOT format) which is In very low birth weight
infants (p),How probiotic administration(I) compared to placebo(C)
reduce the incidence of NEC ? (O)
• Based on these we searched three available free data base which is
PuBMed,Embase and Cochrane database for systematic review
• We used key words from clinical question including : probiotic
administration, very low birth weight infant(VLBW) infant,
necrotizing enter colitis prevention and synonym like:
supplementation of probiotic ,promoting enteral feeding in preterm
were used for searching strategy
• Whenever terms from a database’s own indexing language, or
controlled vocabulary, matched the keywords or synonyms, those
terms were also searched.
• We were also used ancestry method from the original reference

37. Cont.
 When we conduct selection on the search engine,
studies fulfilling primarily the following:
• Type of design of a probiotic regimen versus placebo
or non-placebo control
• Study populations involving VLBW infants (i.e.
neonates with a birth weight of <1500g)
• Outcomes of NEC stage ≥2 (according to Bell’s
staging criteria)
• Papers written in English.
NB: These criteria were considered in addition to rapid
critical check list

38. Cont.
After searching we were get 12 studies from
PubMed,1study from EMBASE and 2 studies from
Cochrane data base of systematic review which are best
for answering the clinical question .
We also used reference lists of each study looking for
any relevant studies we were 1 study found in their
original search(ancestry method).
 Finally we were obtained a total of 16 studies based
on our searching strategy.
 Critical appraisal isn’t solely to find the flaws in a
study, but to determine its worth to practice.

39. Cont.
• Finally after critically appraising each study with
there respective check list we were identified 3
systematic review including meta-analysis , 1
RCT and 1 retrospective cohort which are
evaluated and synthesized for the seek of clinical
decision making.
• After critical appraisal most of the studies
obtained from data base were not reach for
synthesis due to small sample size, lack of recent
<2010 and question to double blinded.

40. Cont.
 Only one study were found that the implementation of routine probiotic
supplementation did not significantly impact the incidence of necrotizing
enter colitis, its severity, or mortality rates in infants with very low birth
weight.
 Three study conducted by systematic review RCT and Meta- a showed that
probiotic provision to VLBW infants were reduced the incidence of NEC
compared to not.
• Measure of effect RR 0.54 [0.45 95%CI, 0.65] showed that probiotic
supplement reduced the incidence of NEC by 46 % compared to with out
providing probiotic
• Another study effect size were RR 0.35 (95 % CI 0.23 – 0.55)
means 65 % of NEC were declined by treatment
• Other review showed the intervention effect were OR 5.7 (95% CI 0.43-
0.74 ) which implies probiotic adoption reducing NEC nearly 6 times than
placebo.
• .....

41. Evaluation table
Citation
of a
Single
Study
Theoretical
or
Conceptual
Framework
Study
Design
and
Method
Sample
Characteristics
and
Setting
Names
and Definitions
of Major
Variables
Outcomes
Measures
Data
Analysis
Findings Level
and
Quality
of
Evidence
Sharif S,
Meader N, 2020
None SR RCT 56trial
n=10842
NICU
DV=NEC
IV=weight,
probiotic and
placebo
RR,95% CI
0.54 [0.45 , 0.65]
Met analysis
Chi-square
SD Level 1
Guthmann , C.
Kluthe , C. B ü
hrer
2010
None MA 4 trial
n=2193
NICU
DV=NEC
IV=weight,
probiotic and
placebo
0.35 (95 % CI
0.23 – 0.55).
Chi-square SD Level 1
John P. Thomas
2015
None MA 23 trial
n=7325
NICU
DV=NEC
IV=weight,
probiotic and
placebo
OR 5.7 (95% CI
0.43-0.74
Cochrane’s Q
test and the
Egger’s
regression test
SD Level 1
Taciana
Duque
Braga2011
None RCT T=231
I=119
C=112
NICU
DV=NEC
IV=weight,
probiotic and
placebo
95% CI
unable to be
calculated (P
= 0.05
Chi-square
test
SD L2
Jessica Que,
Rhonda Van
Oerle,
Susan
Albersheim,
2021
None Cohort T=665
I=310
C=355
‘’ 5% v. 4%, p
= 0.35
χ2 NSD L3

42. Example of Headings for a Synthesis
Table
Study
Author
Year Number of
Participants
Mean
Wt.(gm)
Study
Design
Interventi
on
Major Finding
That Addresses
Your Question
Sharif S,
Meader N
2020 10842 1100 SRRCT probiotic Reduce NEC
Guthmann , C.
Kluthe ,
CBührer
2010 2193 - MARCT probiotic Reduce NEC
John P. Thomas 2015 7325 - MARCT probiotic Reduced NEC
Taciana Duque
Braga
2011 231 1124.5 RCT Bifidobacteriu
m breve and
Lactobacillus
casei
Reduced NEC
Jessica Que,
Rhonda Van
Oerle,
Susan
Albersheim,
2021 665 1090 cohort Bifidobacteriu
m and
Lactobacillus
No significance

43. Conclusion
• Despite the controversy that characterizes this
topic, our analysis shows with considerable
statistical confidence that probiotics prevent
NEC and reduce mortality in VLBW infants.

44. REFERENCE