INTRODUCTION

1. HARAMAYA UNIVERSITY HIWOT
FANA COMPREHESIVE SPECIALIZED
HOSPITAL
CLINICAL PHARMACY AND DRUG
AND POISON INFORMATION
SERVICE (DPIS) UNIT
BY SALAHADIN M.ALI (B.PHARM)
1

2. INTRODUCTION TO LITERATURE
EVALUATION
2

3. Primary literatures
• Original published or unpublished works that
introduce new knowledge or enhance existing
knowledge
• Sources of all literatures
• The majority of clinical decisions are based on
primary literature reports.
However,…….there is need to evaluate this literatures
critically
3

4. Literature evaluation
• Critical literature evaluation is the process of reading
and evaluating primary literature journal articles in
order to arrive at an interpretation that you can call
"your own“.
– Are the results believable?
– Are the results applicable to your practice?
– Does the paper really support it claims?
• The ultimate interpretation and decision about the
value of an article rests with the reader.
4

5. • Many studies are open to differing interpretation
• It has been estimated that 40 to 50% of published
articles in the medical literature have serious
problems with study design, statistical analysis, and
conclusions.
5
Literature evaluation..

6. Why literature evaluation?
• Wrong study design
• Use the right methods incorrectly
• Misinterpret their results
• Reference other studies selectively or incorrectly
• Draw unjustified conclusions from research
• Peer-reviewer's lack of knowledge in study design or
statistical analysis
• Scientific fraud
6

7. Stages Where Bias Can Occur Within a
Study
1. Reading background information or the introduction
of the study
2. Defining and choosing the study sample
3. Applying the experimental interventions
4. Measuring the outcomes
5. Analyzing the data
6. Interpreting the analysis and results
7. Publishing the findings
7

8. Common Biases Found in Research
• One-Sided Reference Bias: Authors may restrict
references to only those works which support their
conclusions.
• Positive Result Bias: Authors are more likely to submit
and publish positive results.
• Sample Size Bias: Samples which are too small can
prove nothing; samples which are too large can prove
anything.
• Volunteer Bias: Volunteers from a specified sample
may exhibit characteristics which may differ from non-
volunteers.
8

9. • Withdrawal Bias: Patients who withdraw from a
study may differ systematically from those who
remain.
• Attention Bias: Subjects may alter their behavior if
they know they are being observed (also known as
the Hawthorne Effect).
• Instrument Bias: Defects in the calibration or
maintenance of measurement instruments may lead
to systematic deviations in results.
9

10. Components of an Evaluative of Journal
Article
Title: A brief description of the article's subject
Authors: Informs the reader who conducted the study;
the primary author or investigator is listed first
Abstract: Identifies the purpose, design, and
methodology of the study and briefly reviews its results
and conclusions
Introduction: Provides a framework for the purpose of
the study through an overview of the literature and
references other published studies that support the
purpose
10

11. Components of an…
Methods: A detailed description of the techniques used
to conduct the study; enough information should be
provided so that the reader could replicate the study.
Results: Organizes and simplifies the raw data, but does
not interpret the results
Discussion: Authors interpret the data and debate the
significance of their findings
Conclusion: Provides a one- to two-sentence summary of
the study's purpose and results
References: Lists all references cited throughout the
study
11

12. Characteristics of Reputable Scientific Journals :
• The quality of the journal an article is published in can
be used as a preliminary, indirect measure of the
potential quality of the article itself.
• An editorial board is one method that helps to ensure
the quality of the information that a particular journal
publishes.
• Peer review is another method employed for helping
ensure the quality of articles published…..Scholars from
diverse institutions should be included
Evaluation of Journal …
12

13. • Editorial policy that specifies the requirements for
the types and formats of submitted manuscripts
• Does not contain an overabundance of
advertisements
Evaluation of Journal …
13

14. Journal impact factor(JIF)
• Journals are ranked according to their impact factor
which is a measure of: 'the frequency with which
the "average article" in a journal has been cited in a
particular year or period‘
• The JIF is calculated considering only those citations
that a particular journal has received within 2 years
prior.
Evaluation of Journal …
14

15. • Impact Factor comparing journal ranks from
disparate disciplines is not valid, due to variations in
publishing cycles, citation rates, the size of the field,
etc.
• The JIF is a measure of journal quality, not quality
of article.
Evaluation of Journal …
15

16. • Other factors to evaluate are the investigators’
credentials and the practice site of these individuals.
• Articles with authors who are employees of a
pharmaceutical company should be more selectively
analyzed, since there may be concern about potential
bias.
• The source of funding and potential conflicts of interest
should be disclosed by the authors.
• Complete funding by pharmaceutical companies may be
an issue of concern
Authors/Investigators
16

17. Evaluating the Title
• The title should be brief and catch the attention of
readers interested in the topic.
• The title should indicate what the article is about
without drawing any conclusions.
• A title that sounds like a newspaper headline may
indicate bias by the authors.
• The title should not give the impression of answering
questions that the study is not designed to answer.
17

18. Evaluating the Abstract
• Abstracts should provide a very brief overview of the
study, and provide enough information for the reader
to determine whether the study is of interest.
• Data presented in the abstract may not be discussed in
the body of the article and important data may be
omitted from the abstract due to space limitations.
• Abstracts should not be used as substitutes for careful
analysis of the study, and clinical decisions should not
be made based on information only from the abstract.
18

19. Evaluating the Introduction
• The introduction should provide valid reasons
supporting the need to conduct the study; all factual
statements must be referenced, and the authors
should appropriately interpret the available literature
for applicability and relevance.
• References should be up-to-date and mostly from the
primary literature- not the tertiary literature. Authors
should avoid citing only their own past research.
• A good introduction should provide a complete
synopsis of the literature published to date and include
the most important studies.
19

20. Evaluating the Introduction…
• Conduct your own literature search to confirm that
the authors have completely reviewed the literature
• The study objective usually appears at the end of the
introduction, and should be stated in a clear and
concise manner.
• If the authors fail to state and objective, it may be an
indication that the study was not well planned.
20

21. • The protocol must be approved the institutional
review board of the institution to ensure patient
safety and sound clinical research
• All subjects must agree to participate by signing an
informed consent that clearly states the risk and
benefits of participating in the study
• The inclusion and exclusion criteria for the subjects
should be appropriate for the topic of study and
clearly defined.
• The methods section should describe in detail how
the patients were selected to participate in the study.
Where is the patient population from?
Evaluating the Methods
21

22. • All measurements should be standardized and
conducted at appropriate intervals.
• Data collection forms and instruments should be
validated.
• Power calculations and level of statistical
significance should be determined a prior
Evaluating the Methods…
22

23. Evaluating the Results
• Data should be presented in a clear and
understandable format.
• Data presented in the abstract, charts, tables, or
graphs should be consistent with what is described in
the text.
23

24. Conclusions/Discussion
• Did the information appear biased or did the trial
results support the conclusions?
• Were trial limitations described?
• Did the investigators explain unexpected results?
• Are the results able to be extrapolated to the
population?
• Were the study results clinically meaningful?
24

25.  An understanding of basic study designs is important
when assessing the validity of literature
 In general, there are two types of study designs:
– Observational studies: where nature determines who
is exposed to the factor of interest and who is not
exposed. These studies demonstrate association
– Experimental studies where the investigator
determines who is exposed. These may prove
causation
Review of different types of study design
25

26. • Observational studies are classified in to two:
descriptive and analytical
– Descriptive studies: Case reports, Case series,
trend studies, cross sectional, etc
– Analytical studies: Case control and cohort
studies
Review of different…
26

27. Commonly encountered biomedical literature
Stability study: Evaluate stability of drugs in various
preparations (e.g., ophthalmologic, intravenous,
topical, and oral)
Cohort (follow-up) study: Determine association
between various factors and disease state development
Case-control study: Determine association between
disease states and exposure to various risk factors
Review of different…
27

28. Cross-sectional study: Identify prevalence of
characteristics of certain factors
Case study or case series: Report observations in a
single patient or series of patient
Bioequivalence study: Assess the bioequivalence of
two or more products
Review of different…
28

29. Post-marketing surveillance study: Evaluate use and
adverse effects associated with newly approved drug
therapies
Clinical trial : Determine cause and effect relationships
Review articles:
– Narrative review: Nonsystematic, subjective
summary of data from multiple studies
– Systematic review: Systematic, qualitative, and
objective summary of data from multiple studies
– Meta-analysis: Combine, statistically evaluate,
and summarize data from multiple studies
Review of different…
29

30. Case study
• A case study —also sometimes referred to as a case
report and referring to a single patient, or a case
series, referring to a group of patients
• Has no control or comparison group, and simply
reports on the clinical course of a particular patient
or group of patients
• Useful for early recognition of drug toxicities and
teratogenicity.
30

31. Cohort study
• Strongest observational study design
• a disease free subject population is divides into two
groups: those identified as either exposed or
unexposed to a factor of interest.
• Subjects are then followed prospectively as
development of a disease state of interest is
observed during the study period.
31

32. Cohort study…
32
Lung disease vs smokers association

33. Evaluation of Cohort study
• The research question must be stated clearly and
unambiguously with relevant inclusion and exclusion
criteria described in detail.
• It is imperative that exposed and unexposed
individuals are similar in terms of demographic…
Cohort study…
33

34. • Information bias can occur if the same efforts to
measure outcomes are not made for both the
exposed and unexposed groups.
• Bias may be further introduced if follow-up rates
differ for the two groups
• Studies where loss to follow-up exceeds 20% in
either the exposed or non-exposed cohort should be
interpreted with caution.
Cohort study…
34

35. Case-control study
• Retrospectively identify potential risk factors of
diseases or outcomes.
• Subjects (cases) with a particular characteristic or
outcome of interest (e.g., disease) are recruited,
matched with, and compared to a similar group of
subjects (controls) who have not experienced the
characteristic or outcome.
35

36. Case-control study…
36

37. Evaluation of Case-control study
• Was predisposition of disease similar in cases and controls
except for exposure to the risk factor?
• Were cases and controls matched?
• Did cases and controls undergo similar diagnostic evaluations?
• Were investigators who assessed patients or collected data
blinded to the status of the subject as a case or control?
• Overall, the two major methodological issues include
appropriate selection of controls and accurate determination
of the level of exposure.
Case-control study…
37

38. Cross-sectional study
• Is kind of “snapshot” because data are collected and
evaluated at a single point in time.
• Problems that may occur during cross-sectional studies;
include errors in data collection and transient effects
that may influence observations.
• Because measurements occur at only one point in time,
inaccuracies in data collection may go unnoticed
because there are no prior data for comparison.
38

39. Evaluation
• Did investigators ensure accuracy in data collection?
• If a survey or questionnaire was used, was it
validated?
• Were the inclusion and exclusion criteria clearly
defined and stated?
• Was selection of cases clearly described?
Cross-sectional study
39

40. Experimental Studies
• It is a study in which a treatment or procedure is
intentionally introduced and a result or outcome is
observed.
• The goal of the clinical trial is to assess the difference
in effect between the investigational and control
groups.
• It could be community based or RCT(randomized
control trial)
40

41. Flow Chart of an Experiment
41

42. Evaluation
• Were reasonable inclusion/exclusion criteria presented
to represent an appropriate patient population?
• Was subject recruitment described? If so, how were
subjects recruited? Was it appropriate?
• Blinding?, Randomization ?
• Were the number of patients withdrawing from the study
reported? Were reasons for subject discontinuations
reported?
Experimental…
42

43. • Were the subject demographics between groups
similar at baseline? If not, were the differences
likely to have an affect on the outcome data?
• Were data presented clearly?
• Were the results statistically significant? Clinically
different?
• Can the trial results be extrapolated to the
population?
Experimental…
43

44. Bioequivalence study
• Because of increasing number of generic products there
is a need to establish the quality, safety, and efficacy of
these generic drugs are the same as the brand name
product.
• Bioequivalence trials are often conducted under
standardized conditions in a small number of normal,
healthy adult volunteers
• A two-treatment crossover study design in 24 to 36
healthy adult subjects is usually used so that the subject
serves as his or her own control
44

45. • Bioequivalent products are products that are
equivalent in rate and extent of absorption (the
rate and extent of absorption differ by ±20% or
less)
Evaluation of Bioequivalence study
• Was age and weight range for the subjects is
defined in the methods and clinical parameters
used to characterize a normal, healthy adult (e.g.,
physical examination observations and hematologic
evaluations) are described.
• All subjects should receive the drug under the same
conditions, and all blood levels should be taken at
the same intervals subject
Bioequivalence study…
45

46. • Subjects should be free of all drugs, including
caffeine, nicotine, and other recreational drugs, for
at least 2 wks prior to testing and usually fast
overnight prior to dosing.
• Methods should define sample collection times, as
well as collection techniques and storage methods of
samples
• Don’t compare two products based on data obtained
from separate studies.
Bioequivalence study…
46

47. Post marketing surveillance
• Once drugs are approved, experience in patients
escalates and previously unrecognized, rare adverse
events may be identified.
• The process of collection of data after marketing is
called post marketing surveillance(Phase IV)
• It is useful in identifying new, potentially serious
effects of drugs.
47

48. Evaluation
• Was a large enough sample studied to reflect current
uses of and side effects associated with the new drug
therapy?
• Were appropriate methods used to measure clearly
defined endpoints?
• Data may be underreported because it is a voluntary
system and this may bias the estimation of incidence,
reports may vary in quality and thoroughness
Post marketing surveillance
48

49. Review Articles
• Attempt to combine results from multiple studies,
generally termed review articles
• Classified as tertiary literature, although they are
often used as secondary sources because they can
lead readers to primary literature references
• The term review encompasses three very different
entities….the nonsystematic (narrative) review, the
systematic (qualitative review), the meta-analysis
(quantitative review)
49

50. • Specific criteria should be used to select articles from
the primary literature to be included in the review.
• Authors must clearly define the study population or
topic of interest and include only those studies using
valid research methods.
• In order to reduce selection bias, review authors
choosing articles should be blinded to
– names of the study authors , institution of
publication, results of the studies
Review Articles…
50

51. Narrative (non-systematic)
• Generally do not apply systematic methods such as
formal criteria for selection of studies, and they
address broad rather than focused clinical questions.
• They often educate readers about the author’s
interpretations of selected evidence, rather than
using a systematic approach to evidence evaluation.
• Frequently, authors are experts on the topic and
know the conclusions prior to conducting the review.
Review Articles…
51

52. Qualitative (systematic) reviews
• Use formal criteria for trial selection and
interpretation of study results
• Authors determine the conclusions based on the
data reviewed.
• Authors should concentrate on a clearly defined
issue that is of importance to practice.
Review Articles…
52

53. Review Articles…
53

54. Meta-analysis
• Meta-analysis is a technique that has been
developed to provide a quantitative and objective
assessment.
• In a meta-analysis, results of previously conducted
clinical trials are combined and statistically
evaluated.
• Meta-analyses are designed to provide greater
insight into clinical dilemmas than individual clinical
trials
Review Articles…
54

55. • Useful when previous studies have been inconclusive or
contradictory, or in situations where sample size may
have been too small to detect a statistically significant
• The quality of the meta-analysis depends on the quality
of the individual studies
• Details of literature searches that were conducted to
locate primary research articles must be given
• Criteria for inclusion of studies in the meta-analysis must
be determined prior to conducting the analysis.
Review Articles…
55

56. 56

57. • IN GENERAL: Interpretation of meta-analyses results
are limited by what studies were (or were not)
included, how homogeneous (or heterogeneous) the
studies were, and the methodological quality of the
studies
• Although the purpose of review articles is to present
the “truth” found among conflicting and variable
primary literature, this does not always occur.
AUTHORS may be subject to biases or inaccuracies
Review Articles…
57

58. Assignment
Select one recently published original research
and evaluate using literature evaluation
criteria's
58

59. Reference
• C.R. Kothari, Research Methodology, Methods
and Techniques, 2nd revised edition, 2004.
59

60. THANK YOU!!!
60