This document discusses the importance of keeping up to date with medical literature for physicians. It notes that over 10,000 new articles are published per week, making it impossible for doctors to read everything. The document then provides guidance on critically evaluating medical literature, including understanding study designs and assessing validity, results, and applicability. It emphasizes applying a systematic approach to identify relevant information and avoid bias. Specific guidance is provided on appraising different study types, such as randomized trials, diagnostic tests, systematic reviews, cohort studies, and case-control studies.

1. Literature EvaluationLiterature Evaluation
Dr.Dr.Ramesh ParajuliRamesh Parajuli
MS (Otorhinolaryngology, Head and Neck Surgery)MS (Otorhinolaryngology, Head and Neck Surgery)
Chitwan Medical College Teaching Hospital, Bharatpur-10,Chitwan, NepalChitwan Medical College Teaching Hospital, Bharatpur-10,Chitwan, Nepal

2. Reasons for Surgeons to Read Medical
Literature:
1.Improve Patient Care
2.Learn about Research
3.Educate Peers and Students about Clinical Care

3. To ensure Best Practice and Treatment for Patients
Doctor must :-
• Access Medical Information
• Acquire New Knowledge
• Achieve Information Mastery in their field
• Without systematic approach to identify & critically appraise clinical research,
doctor might become dependant on inappropriate or outdated information

4. Keeping Medical Informations up to date
Past:
• Reading few journals every month
• Occasional conferences
Present:
• Internet
• Information- available to everyone

5. • Over 10,000 new articles per week are archived
by the National Library of Medicine, and at least twice
that many are published worldwide. (Rudmik LR,2008)
• More than 2 millions/year articles
published in the biomedical press
(scott Brown 7th
edn)
• Most published articles are not translatable
to clinical practice, or
their methods are not clinically sound

6. Publication Bias:-
Articles with Positive findings: Easier to get published
Articles with Negative findings: Author/Editors
• Publication Bias Biased sample of article

7. • As many as 30% of journal articles may contain errors.
(Victor A F,2003)
• Among health professionals, medical literature is the
preferred source of new knowledge
• Medical informations doubles every 5 years
(Castillo D L)

8. To keep up with available clinical evidence, today’s
dedicated physician would have to read 19 articles
everyday, of which only15% would provide information
of practical relevance. (Diana L C,2008)

9. Journal Article
• Rigid structure
• Extracting information easier
Format:
1.Abstract
2.Intorduction
3.Methods
4.Results
5.Discussion
6.Conclusion

10. 1.Abstract–overview of
• What the research is about
• What it did
• How it did
• What is found
• What those results mean
2.Introduction –
• Review of previous research
• Rationale for the research
• What the research is aiming to do
3.Method:
• Methods used in the Research

11. 4.Result :
• Describe the result found
5.Discussion:
• What the results actually mean
6.Conclusion:
• Results of several experiments presented and discussed,
Implications of the research
• Recommendations about further research or policy and
practice in the relevant area

12. Knowledge of the Structure and Function of the
Sections of Journal Article:-
• Reduces the time to locate information
• Improves efficiency of reading
• Reduces the time to read the article

13. • Reading Articles Uncritically
Acceptance of Text and Arguments
Flawed, Biased and Subjectively written
• Efficient Reading is not the only skill we need Critical Reading

14. Critical Reading:
Suspending the Judgment on a text until
1.Have understood the message being put forward
2.Evaluated the evidence supporting that message
3.Evaluated the writer’s perspective

15. Critical Reading Checklist
•To avoid reading poor quality literature andTo avoid reading poor quality literature and
mastering irrelevant informationmastering irrelevant information SystematicSystematic
StrategyStrategy
•A skill that a busy otolaryngologist can learnA skill that a busy otolaryngologist can learn
•Critical reading check listCritical reading check list for critical readingfor critical reading

16. Critical Reading for Research Articles
• Are the limitations of the procedures clear?
• Is the methodology valid? (eg.sample size, sampling method)
• Are the results consistent with the objectives?
• Are the claims the author makes about his or her own
research internally consistent?
• Are the diagrams clear to the reader?

17. • Critical Thinking/Critical Analysis:
””thethe intellectually disciplined process of actively and skillfully
conceptualising, applying, analysing, synthesising and/or evaluating
informations gathered from ,or generalised by, observation,
experience, reflection, reasoning or communication ,as a guide to
belief or action (or argument)” (Scriven & Paul 2001)

18. What critical analysis is NOT:
1.A summary of a text
2.A descriptive comment on the text
3.A rehash of the theory learnt in a class
4.A statement of one’s own unsupported views

19. • Harm to pts can be aboutHarm to pts can be about:-:-
Diagnosis:-Diagnosis:-
Therapy:-Therapy:-
Prognosis:-Prognosis:-
• To prevent the harm to the pts:-mostTo prevent the harm to the pts:-most up to dateup to date
knowledgeknowledge availableavailable

20. • Evidence Based Medicine-”evidence-based medicine is the
conscientious, explicit and judicious use of current best
evidence in making decisions about the care of individual
patients” (Sackett et al. 1996 )

21. • CONSCIENTIOUS
- To do more good than harm
- To do right things in right way
- Carefulness and attention
• EXPLICIT
- Openness and transparency
-To evaluate and criticize
the methods used
- Doctor- Patient relationship
• JUDICIOUS
- To evaluate and weigh
the evidence of the work
-To consider Benefit &
balance it against harm
• Current Best Evidence

22. Evidence Based Practice (EBP)
• Begins and ends with patients
• Integrating individual clinical expertise with the best available
external clinical evidence from systematic research
• Application of the knowledge to the prevention, diagnosis or
management of the disease

23. Evidence Based Medicine –practice of medicine with an
emphasis relying on the medical literature for clinical
decision making
EBM approach-
• Improves ability to evaluate clinical literature
• Enhances life long learning skills in day to day medical practice
• Keeping up to date
• Continuing medical education (CME)

24. Purpose of EBM-
• To propagate the evidence but not to replace individual’s
clinical experience
• One shouldn’t expect clinical practice to be 100%evidence
based-better patient care shouldn’t be compromised by not
counting on evidence when it is readily available by different
sources.

25. • EBM is key component of modern medical practice
• EBM is necessary but not sufficient
• Otolaryngologists should acquire the fundamentals skills
of searching, appraising, and synthesizing, required to
practice EBM and should do so with the same diligence
they apply to learning surgical skills

26. Critical Appraisal –one aspect of EBM
• The ability to apply the principals of analysis
• To identify those studies which are unbiased and valid.
Critical Appraisal of Research Studies helps to:-
1.Improve patient care
2.Improve teaching
3.Become a better researcher
4.Prepare for journal club
5.Gain respect among the peers

27. Practising Evidence Based Health Care:-
EBM process involves 5 steps
1.Ask clinical question:-formulation of “P I C O” question
2.Search literature
3.Appraisal (critically evaluate the literature)
4.Share the knowledge
5.Apply to the patient

28. • Forming Answerable Questions:
1.Background questions:-understand the problem in general
2.Foreground questions:-decision making questions

29. • The PICO Format for Foreground Questions
1.P=Patient and Problem (population-kids, women, patients)
2.I=Intervention( test)
3.C=Comparison intervention (Control group)
4.O=Outcome

30. • To avoid reading poor quality literature and mastering irrelevant
information Systematic Approach
1.Retrieve
2.Review
3.Reject
4.Read

32. • 5s model for Evidence-Based Health Care
Decisions (Hynes R B; ACP J Club 2006;145:A8)

33. To answer a particular question: evidence Based Research
Resources
1.The cochrane collaboration
2.National library of medicine-gateway
3.Open clinical
4.bestBETs
Selection of medical database:
• Type of question
• Ease of use for a particular problem
• Need to limit the search for highest quality studies

34. • Selecting an Article: Filtering Process
1.Primary Survey (initial evaluation and brief overview)
(A) Analyse the Title :
(B) Review the list of authors:
(C) Read the summary or abstract beginning with conclusion:
• If the conclusion if valid ,important to me?
• Results if true, how useful they are?
• Is the primary outcome measured important for me?
• Do the interventions make sense?
• Can the information be generalised to my patients?

35. Secondary Survey:
1.Introduction :
• Problem under study, context of the study, reasons for conducting
study
• Importance of the topic, what is known and what is unknown about
the topic
• Specific questions (objective, goal of the study, hypothesis) to be
evaluated
• Study sample, primary outcome & intervention being evaluated
• Method design
• Conclusion shouldn’t extend beyond the stated objective

36. 2.Methods:
• Research design-descriptive or comparative study
• Single or multicenter
3. Study Sample:
• How were the subjects and controls selected?
• Are the inclusion & and exclusion criteria sufficiently clear to
describe the target population?
4.Treatment Allocation:
• Randomization
• Masking (blinding)

37. 5.Outcomes:
primary outcome-all studies
secondary outcome-some studies
• How it was measured?
• was the measurement free of bias ?
• How reproducible were the results?
• How to standardize measurements & to minimize inter-observer variability?
6.Statistical Analysis:
• Statistical Tests

38. 7.Results:
• Tables and Figures,
• How many pts eligible for the study?
• How many enrolled?
• How many completed?
8.Discussion :
• Author’s interpretation of clinical relevance
• Comparison with previous studies, similarities & differences
• Limitation of the study
• Suggest new directions for appropriate study

39. 9.Conclusion:
• Must consistent with the study objective
• Justified by the study results
• Shouldn’t over generalize the results of the study

40. • If there is not concealment of Randomizationmay
exaggerate efficacy of the treatment by as much as 30% more
• If there is no Blinding may exaggerate the effectiveness of
the treatment by 15%
• If <80% pts Followed Upresults become meaningless

41. Types of Study by Content:
1.Evaluation of a new therapy
2.Evaluation of a new diagnostic test
3.Determinnation of the etiology of a condition
4.Prediction of the outcome
5.Natural course of a condition

42. Levels of evidence
• Level 1 : 1a-SR(homogeneity) of RCTs
1b-Individual RCT
1c-All or None
• Level 2 :2a-SR(with homogeneity) of Cohort studies
2b-Individual Cohort Study (including low quality RCT; eg
<80% follow up)
2c-Outcomes research; ecological studies
• Level 3 3a-SR(with homogeneity) of case control studies
3b-Individual case-control study
• Level 4 - Case Series(& poor quality cohort & case control studies)
• Level 5 - Expert opinion

43. Grades of Recommendation
• Grade A- consistent level 1 studies
• Grade B- consistent level 2 or 3 studies or
extrapolations from level 1 studies
• Grade C- level 4 studies or extrapolations
from level 2 or 3 studies
• Grade D- level 5 evidence or troublingly
inconsistent or inconclusive studies of any
level

44. once the trial or trials have been found that
seems to answer the questions check three
things:
1.Is the study valid?
2.What are the results?
3.Can these be applied to my patients?

45. (I) Appraising the Diagnostic Articles:
1.Is the study valid?
Did the authors answer the questions?
1.What were the characteristics of the groups?
2.Is it clear how the test was carried out?
3.Is the test result reproducible?
4.Was the reference standard (gold standard) appropriate?
5.Were the reference standard & the diagnostic test interpreted blind
and independently of each other?
6.Was the reference standard applied to all patients?
7.Was the test evaluated on an appropriate spectrum of patients?

46. 2. What were the results?
• Are the sensitivity/specificity and/or likelihood ratios presented?
• Could the results have occurred by chance?
• Are there confidence limits?
Likelihood Ratio :-
• Likelihood Ratio (+ve results)= sensitivity
(100% - specificity)
• Likelihood Ratio (-ve results)= (100% - sensitivity)
specificity
If LR > 10 + ve
< 1 - ve

47. 3. Will the Results help my Patients?
1.Is the diagnostic test available, affordable, accurate & precise in
my setting?
2.Are the results applicable to my pts? Do my pts have a similar
mix of disease severity & competing conditions?
3.Will the results change my case management ?will the
information gain be sufficient to change a clinical decision (rule
in or rule out)?
4.Will patients be better off as a result of performing the test?

48. (II) Appraising Therapy Articles
1.IS THE STUDY VALID?
Did the authors answer the questions?
1.What were the characteristics of the pts?
2.Were the groups similar at the start of the trial?
3.Aside from the experimental treatment, Were the groups treated equally?
4.What was the treatment & what was it compared against placebo?
5.Was randomization used?
6.Were all pts followed up at its conclusion?
7.Wre they analyzed in the groups to which they were randomized?
8.Were blinding used?
9.Was the length of study appropriate?

49. 2.WHAT WERE THE RESULTS?
Event rates-control event rate (CER), experimental event rate (EER)
Relative risk
CER-EER
Relative risk reduction (RRR) = ------------------
CER
Absolute risk reduction (ARR) = CER- EER
Number needed to treat (NNT) = 1
-------
ARR

50. 3.WILL THE RESULTS HELP MY PATIENTS?
1.Can the results be applied to my patient care?
Would my pts eligible for the study?
Are there any reasons why the results shouldn’t be applied to my
pts?
2.Were all clinically important outcomes consider?
3.Are the likely treatment benefits worth the potential harm and
costs?

51. (III) Appraising the Systematic Review
1.IS THE REVIEW VALID?
Did the authors answer the questions?
1.What databases & other sources did the authors of this review search?
2.What were their eligibility criteria (inclusion & exclusion)? do these seems
appropriate?
3.Was there independent data extraction of the result by the reviewers (then
compared later)?
4.Is there a description of the quality of each trial?
5.Were the results consistent from study to study (homogeneous)?

52. (2)WHAT Were The Results?
If the results of the studies have been combined, was
it reasonable to do so?
consider whether:
• the results of each study are clearly displayed?
• the results were similar from study to study?
How are the results presented and what is the main
result?
How the results are expressed (eg. odds ratio, relative risk etc)
How precise are these results?

53. 3 .CAN I APPLY EVIDENCE FROM THIS SYSTAMATIC REVIEW IN
CARING FOR MY PTS?
• Whether the population sample different from my population ?
• Whether my local setting differs ?
• Were all clinically important outcomes considered?
• Should policy or practice change as a result of the evidence
contained in this review?
• Whether any benefit reported outweighs any harm/or cost

54. (IV) Critical Appraisal Skills for the Cohort Study
1.ARE THE RESULTS VALID?
• Did the study address a clearly focused issue?
• Did the authors use an appropriate method to answer their
question?
• Was the cohort recruited in an acceptable way?
• Was the exposure accurately measured to minimize bias?
• Was the outcome accurately measured to minimize bias?

55. • Have the authors identified all important confounding
factors?
• Was the follow up of subjects complete enough?
• was the follow up of subjects long enough?
2.WHAT ARE THE RESULTS?
• What are the results of this study?
• How precise are the results?
• Do I believe the results?

56. 3.WILL THE RESULTS HELP ME LOCALLY?
• Can the results be applied to the local population?
• Do the results of this study fit with other available evidence?

57. (V) Appraisal of a Case Control Study
1.ARE THE RESULTS OF THE STUDY VALID?
1.Did the study address a clearly focused issue?
2.Did the authors use an appropriate method to answer their question?
3.Were the cases recruited in an acceptable way?
4.Were the controls selected in an acceptable way?
5.Was the exposure accurately measured to minimize bias?
6.(A)what confounding factors have the authors accounted for?
(B) Have the authors taken account of the potential confounding factors in
the design and/or in their analysis?

58. 2.WHAT ARE THE RESULTS?
7.What are the results of this study?
8.How precise are the results?
how precise is the estimate of risk?
9.Do I believe the results?

59. 3.WILL THE RESULTS HELP ME LOCALLY?
10.Can the results be applied to the local population?
11.Do the results of this study fit with other available evidence?

60. REFERENCES:
1.Scott-Brown’s Otorhinolaryngology, Head & Neck Surgery(7th
edition)
2.Critical appraisal skills programme (CASP)-making sense of evidence
public health resource unit;england(2006)
3.Evaluating the literature (emedicine)
4.Unilearing-University of wollongong, australia
5.Evidence based medicine (EBM)-what, why and how(KUMJ 2003)
6CEBM-EBM tools
7.Assesing the medical literature: let the buyer beware
8.Knoweledge managemet:how to keep up with the literature
9.Literature review(st.kate’s libraries guides)
10.How to evaluate the literature(stanford university)
11.Guidelines for evaluating writing about literature
(john jay college of criminal justice)
12.Critical analysis-so what does that mean(university of bradford)
13.Evaluating the medical literature(clista clantin 2009)
14.Literature evaluation (hussain al awami)

61. 16.How to read a paper? (BMJ)
17.Crafting the literature review (massey university;2006)
18.Argument based medical ethics
19.Assessment of the critical appraisal (american journal of
surgery,2004)
20.Teaching critical appraisal skills to medical students in obstetrics and
gynecololgy (university of california OBGYN department)
21.A simple method for evaluating clinical literature (JF Robert)
22.Evidence based medicine literature evaluation questions (washington state
university,2006)
23.Evidence based medicine capitol conference 2007

62. Thank you