EVALUATION OF RELEVANT SCIENTIFIC LITERATURE Ahmed Zeeneldin Associate Prof of Medical OncologyMaster of Clinical Research, Liverpool, UK
Why should we evaluate the literature?• PubMed • 2011: ~ 1000 000 new articles added • Dec 2011: >100 000 new articles added • Anthracycline AND chemotherapy AND breast cancer: >8000 articles• Thus: • little time and huge literature volume • Prioritize • Evaluate what you read/hear
Acquiring information• The ‘push’ method • lectures, seminars, • reading journals • no control over the content• The ‘pull’ Method • You search for answers for your questions
This medicine is based on what? Based on Nothing?
Evidence 3-dimensions (apq)• Author (a) • Known • Expert• Publisher (p) • Peer-reviewed • Non-reviewed• Quality of evidence (q) • On the fly: EBM guides • Comprehensive
Literature assessment/appraisal• Pre-appraised sources • Up-to-date • Trip database • Rehab+ database • Risk lies on expert authority (a & P)• Appraise it yourself (q) - Rapidly: EBM tools - Extensively: critical appraisal tools
Types of Information Sources• Primary Sources: original research • Not appraised• Secondary Sources: about primary • Appraised• Tertiary Sources: about secondary • Appraised
Primary Sources• Original materials• Not been filtered through interpretation, condensation• Not evaluated by a second party;• Examples: • Original journal articles • Theses
Secondary Sources• Sources about primary, or original, information,• Modified, selected, or rearranged for a specific purpose or audience• Examples: • Textbooks • Review articles
Tertiary Sources• Sources twice removed from original sources• Examples: • Handbooks • Encyclopedias
Steps of reading literature• You may start by 3ry or 2ry information recourses • Textbook • Review article • Encyclopedia • handbook• To get overview of the topic• Then you use the bibliography to locate 1ry sources• Or search directly for 1ry sources
Evidence-based medicine (EBM) tools• EBM: is a process to systematically • FIND, • APPRAISE, and • APPLY • research findings to clinical decisions
Evidence Pyramid• I - Controlled and randomized• II-I Controlled but not randomized• II-2 Cohort or case control• II-3 Multiple time series• III- Expert opinion or case study
2011 OXFORD CENTRE FOR EVIDENCE-BASED MEDICINE (OCEM) LEVELS OF EVIDENCE EVIDENCE BOXES
2011 OCEBM Levels of Evidence (Evidence Boxes)If you have limited time, where do you beginsearching for evidence? - hierarchy of the likely best evidence - for busy people (few minutes to few hours)
PubMed search for “ Anthracycline AND chemotherapy AND (breast cancer) ” plus some filtersType Term used No. of articlesAll articles (no filter) 8066SR Systematic review 32RCT "random allocation" [MeSH] 120DBRCT double blind method [mh] 58Cohort "cohort studies" [MeSH] 1379Case-control "Case-Control Studies"[Mesh] 521Case report Case Reports [Publication Type] 568
Evidence according to question• Incidence (How common is the problem?) • Local surveys with random sampling OR consensus• Diagnosis (Is this diagnostic or monitoring test accurate?) • SR of cross sectional studies with reference standard• Prognosis (What will happen if we do not add a therapy?) • SR of inception cohort studies• Treatment Benefits (Does this intervention help or harm?) • SR of RT• Screening (Is this (early detection) test worthwhile • SR of RT
Critical appraisal of study reports• Systematic Review• Diagnostics• Prognosis• RCT•
Systematic Review Appraisal• What is the question (PICO)?• Were important and relevant studies missed?• Were criteria to select articles appropriate?• Were the quality of included articles assessed?• Were the results similar from study to study?• What were the results? and how they are presented?
Diagnostic test study appraisal• Was the diagnostic test evaluated in a representative spectrum of patients?• Was the reference standard applied regardless of the index test result?• Was there an independent, blind comparison between the index test and an appropriate reference (gold) standard of diagnosis?• Are test characteristics presented? • Accuracy: sensitivity and specificity • Performance in population: PPV and NPV• Details to replicate the test?
Critical appraisal of Prognostic studies• Were patients representative and assembled at a common point?• Was patient follow-up sufficiently long and complete?• Were outcome criteria either objective or applied in a ‘blind’ fashion?• If subgroups with different prognoses are identified, did adjustment for important prognostic factors take place?• How likely are the outcomes over time?• How precise are the prognostic estimated?• Are the results applicable to my patients?
RCT appraisal• What question did the study ask (PICO)?• Was the assignment of patients to treatments randomised?• Were the groups similar at the start of the trial?• Aside from the allocated treatment, were groups treated equally?• Were all patients who entered the trial accounted for? and were they analysed in the groups to which they were randomised?
RCT appraisal• Were measures objective or were the patients and clinicians kept “blind” to which treatment was being received?• How large was the treatment effect?• How precise was the estimate of the treatment effect?• Will the results help me in caring for my patient?
Steps of EBM 1. asking questions 2. searching for answers 3. critically appraising the results _____________________________ 4. determining applicability to practice 5. integrating data with experience 6. involving patient in decisions
Asking questions 1) what is the question about? • Therapy, diagnosis, prognosis, aetiology/harm, screening etc 2) what is the type of question? - background Q (topic) - foreground Q (patient) 3) how can questions be structured to facilitate a search for precise answers?PICO’ components: • Patient group/Population • Intervention • Comparison • Outcome
Examples of Focused Questions 1 2 3 4 Patient or Intervention Comparison Outcomes (a cause, prognostic Problem factor, treatment, etc.) Intervention (if necessary) In patients with MBC does FEC regimen Compared to CMF Yield better OSMed Oncology regimen rates? In patients with early BC Does BCS Compared to MRM Lead to higherSurgical recurrences andOncology lower OS? In patients with operable Does IO RT Compared to PO RT Yield higherRadiotherapy BC recurrences? In patients with ovarian Does HEP4 Compared to CA125 Leads to earlierClinical cancer diagnosis ofPathology recurrence? In normal people Does smoking Compared to no Increase risk ofEpidemiology smoking lung cancer
How to Conduct a Computer Search1. Specify the research problem 1. Honey and cancer2. Select the databases 2. PubMed3. Select the key concepts, 3. Honey AND cancer operators, limits 4. ENTER: 198 records4. Conduct the search 5. Honey OR bees5. Increase or decrease your total AND cancer: 281 results Honey AND cancer6. Review the citation list AND treatment: 129
Determining the Level of Evidence• Quickly: • Evidence box according to question• Thoroughly (critically appraise according to study type): • Relevance (related): • applicability of the evidence to my problem • Validity (correct): • the extent to which a concept, conclusion or measurement is well- founded and corresponds accurately to the real world
TRANSLATINGEVIDENCE INTODECISIONThe integration of relevant evidence withclinical experience
Trial Validity• Internal: proper study • Did the study measure what it is meant to? • Do we trust the results?• External: generalizability of results
Making decisions• Define the population and intervention • Patients’ characteristics • Intervention: price and availability, training• Search for and understand the biases • Critical appraisal• Interpret the findings and apply them to your patient • Sample size • Summary statistic: • Probablity (p) and confidence of the results (CI)
Conclusion• Pre-appraised literature may be the main source of information for undergraduate students• However, graduates and faculty staff should be able to appraise information both quickly (EBM tools) and thoroughly (critical appraisal tools)• APQ 3-dimension concept
Recommendation• I suggest that principles of EBM be taught to all NCI students• By whom? • Concerned Departments? • Epidemiology/Biostatistics Department?