This is a rapid guide for assessment of literature (EBM)

1. EVALUATION OF RELEVANT
SCIENTIFIC LITERATURE
Ahmed Zeeneldin
Associate Prof of Medical Oncology
Master of Clinical Research, Liverpool, UK

2. Why should we evaluate the literature?
• PubMed
• 2011: ~ 1000 000 new articles added
• Dec 2011: >100 000 new articles added
• Anthracycline AND chemotherapy
AND breast cancer: >8000 articles
• Thus:
• little time and huge literature volume
• Prioritize
• Evaluate what you read/hear

3. Acquiring information
• The ‘push’ method
• lectures, seminars,
• reading journals
• no control over the content
• The ‘pull’ Method
• You search for answers
for your questions

4. This medicine is based on what?
Based on Nothing?

5. Evidence 3-dimensions (apq)
• Author (a)
• Known
• Expert
• Publisher (p)
• Peer-reviewed
• Non-reviewed
• Quality of evidence (q)
• On the fly: EBM guides
• Comprehensive

6. Literature assessment/appraisal
• Pre-appraised sources
• Up-to-date
• Trip database
• Rehab+ database
• Risk lies on expert authority (a & P)
• Appraise it yourself (q)
- Rapidly: EBM tools
- Extensively: critical
appraisal tools

7. Types of Information Sources
• Primary Sources: original research
• Not appraised
• Secondary Sources: about primary
• Appraised
• Tertiary Sources: about secondary
• Appraised

8. Primary Sources
• Original materials
• Not been filtered through interpretation, condensation
• Not evaluated by a second party;
• Examples:
• Original journal articles
• Theses

9. Secondary Sources
• Sources about primary, or original, information,
• Modified, selected, or rearranged for a specific purpose or
audience
• Examples:
• Textbooks
• Review articles

10. Tertiary Sources
• Sources twice removed from original sources
• Examples:
• Handbooks
• Encyclopedias

11. Steps of reading literature
• You may start by 3ry or 2ry information recourses
• Textbook
• Review article
• Encyclopedia
• handbook
• To get overview of the topic
• Then you use the bibliography
to locate 1ry sources
• Or search directly for 1ry sources

12. Evidence-based medicine (EBM) tools
• EBM: is a process to systematically
• FIND,
• APPRAISE, and
• APPLY
• research findings to clinical decisions

13. OLD EBM CONCEPT

14. Evidence Pyramid
• I - Controlled and randomized
• II-I Controlled but not randomized
• II-2 Cohort or case control
• II-3 Multiple time series
• III- Expert opinion or case study

15. 2011 OXFORD CENTRE FOR EVIDENCE-BASED
MEDICINE (OCEM) LEVELS OF EVIDENCE
EVIDENCE BOXES

16. 2011 OCEBM Levels of Evidence (Evidence Boxes)
If you have limited time, where do you begin
searching for evidence?
- hierarchy of the likely best evidence
- for busy people (few minutes to few hours)

17. PubMed search for
“ Anthracycline AND chemotherapy AND (breast cancer) ” plus some filters
Type Term used No. of articles
All articles (no filter) 8066
SR Systematic review 32
RCT "random allocation" [MeSH] 120
DBRCT double blind method [mh] 58
Cohort "cohort studies" [MeSH] 1379
Case-control "Case-Control Studies"[Mesh] 521
Case report Case Reports [Publication Type] 568

18. Evidence according to question
• Incidence
(How common is the problem?)
• Local surveys with random sampling OR consensus
• Diagnosis
(Is this diagnostic or monitoring test accurate?)
• SR of cross sectional studies with reference standard
• Prognosis
(What will happen if we do not add a therapy?)
• SR of inception cohort studies
• Treatment Benefits
(Does this intervention help or harm?)
• SR of RT
• Screening
(Is this (early detection) test worthwhile
• SR of RT

19. Critical appraisal of study reports
• Systematic Review
• Diagnostics
• Prognosis
• RCT
•

20. Systematic Review Appraisal
• What is the question (PICO)?
• Were important and relevant studies missed?
• Were criteria to select articles appropriate?
• Were the quality of included articles assessed?
• Were the results similar from study to study?
• What were the results? and how they are
presented?

21. Diagnostic test study appraisal
• Was the diagnostic test evaluated in a representative
spectrum of patients?
• Was the reference standard applied regardless of the
index test result?
• Was there an independent, blind comparison between
the index test and an appropriate reference ('gold')
standard of diagnosis?
• Are test characteristics presented?
• Accuracy: sensitivity and specificity
• Performance in population: PPV and NPV
• Details to replicate the test?

22. Critical appraisal of Prognostic studies
• Were patients representative and assembled at a
common point?
• Was patient follow-up sufficiently long and complete?
• Were outcome criteria either objective or applied in a
‘blind’ fashion?
• If subgroups with different prognoses are identified, did
adjustment for important prognostic factors take
place?
• How likely are the outcomes over time?
• How precise are the prognostic estimated?
• Are the results applicable to my patients?

23. RCT appraisal
• What question did the study ask (PICO)?
• Was the assignment of patients to treatments
randomised?
• Were the groups similar at the start of the trial?
• Aside from the allocated treatment, were groups treated
equally?
• Were all patients who entered the trial accounted for?
and were they analysed in the groups to which they were
randomised?

24. RCT appraisal
• Were measures objective or were the patients and
clinicians kept “blind” to which treatment was being
received?
• How large was the treatment effect?
• How precise was the estimate of the treatment effect?
• Will the results help me in caring for my patient?

25. Steps of EBM
1. asking questions
2. searching for answers
3. critically appraising the results
_____________________________
4. determining applicability to practice
5. integrating data with experience
6. involving patient in decisions

26. Asking questions
1) what is the question about?
• Therapy, diagnosis, prognosis, aetiology/harm, screening etc
2) what is the type of question?
- background Q (topic)
- foreground Q (patient)
3) how can questions be structured to facilitate a search
for precise answers?
PICO’ components:
• Patient group/Population
• Intervention
• Comparison
• Outcome

27. Examples of Focused Questions
1 2 3 4
Patient or Intervention Comparison Outcomes
(a cause, prognostic
Problem factor, treatment, etc.) Intervention
(if necessary)
In patients with MBC does FEC regimen Compared to CMF Yield better OS
Med Oncology
regimen rates?
In patients with early BC Does BCS Compared to MRM Lead to higher
Surgical
recurrences and
Oncology
lower OS?
In patients with operable Does IO RT Compared to PO RT Yield higher
Radiotherapy
BC recurrences?
In patients with ovarian Does HEP4 Compared to CA125 Leads to earlier
Clinical cancer diagnosis of
Pathology recurrence?
In normal people Does smoking Compared to no Increase risk of
Epidemiology
smoking lung cancer

28. How to Conduct a Computer Search
1. Specify the research problem 1. Honey and cancer
2. Select the databases 2. PubMed
3. Select the key concepts, 3. Honey AND cancer
operators, limits
4. ENTER: 198 records
4. Conduct the search
5. Honey OR bees
5. Increase or decrease your total AND cancer: 281
results Honey AND cancer
6. Review the citation list AND treatment: 129

29. Determining the Level of Evidence
• Quickly:
• Evidence box according to question
• Thoroughly (critically appraise according to study
type):
• Relevance (related):
• applicability of the evidence to my problem
• Validity (correct):
• the extent to which a concept, conclusion or measurement is well-
founded and corresponds accurately to the real world

30. TRANSLATING
EVIDENCE INTO
DECISION
The integration of relevant evidence with
clinical experience

31. Trial Validity
• Internal: proper study
• Did the study measure what it is meant to?
• Do we trust the results?
• External: generalizability of results

32. Making decisions
• Define the population and intervention
• Patients’ characteristics
• Intervention: price and availability, training
• Search for and understand the biases
• Critical appraisal
• Interpret the findings and apply them to your patient
• Sample size
• Summary statistic:
• Probablity (p) and confidence of the results (CI)

33. Conclusion
• Pre-appraised literature may be the
main source of information for
undergraduate students
• However, graduates and faculty staff
should be able to appraise
information both quickly (EBM tools)
and thoroughly (critical appraisal
tools)
• APQ 3-dimension concept

34. Recommendation
• I suggest that principles of EBM be taught to all NCI
students
• By whom?
• Concerned Departments?
• Epidemiology/Biostatistics Department?