Research Methods 2 Critical Appraisal Of Literature
CRITICAL APPRAISAL OF LITERATURE
Why do we need to appraise the available literature?
> Volume of information available for patient management > Variety of evidence available on one topic (e.g., books, journals) > Want to give the best possible treatment to our patients
OBJECTIVES 1. To understand what is evidence-based medicine (EBM) 2. To apply the rule of evidence in order to decide on the validity of an article on therapy / diagnostic test
EVIDENCE-BASED MEDICINE (EBM) <ul><li>new paradigm in clinical decision-making </li></ul><ul><li>OLD PARADIGM: </li></ul><ul><li>> observations from clinical practice </li></ul><ul><li>> basic disease mechanisms and pathophysiologic principles </li></ul><ul><li>> traditional medical training & common sense </li></ul><ul><li>> content expertise & clinical experience: guidelines for clinical practice </li></ul>
> NEW PARADIGM: > clinical experience: important & necessary, but not the only elements of becoming a competent physician > understanding of basic disease mechanisms are necessary, but insufficient > understanding rules of evidence: correct interpretation of available literature > application of the rules of evidence to evaluate the available clinical literature to answer a clinical dilemma
CLINICAL SCENARIO <ul><li>45/M, smoker </li></ul><ul><li>government employee </li></ul><ul><li>would like to stop smoking, but does not want to use any pharmacologic intervention </li></ul><ul><li>given health education and advised to follow-up </li></ul><ul><li>DILEMMA: Among adult smokers, what is the effectiveness of counseling in smoking cessation using a randomized controlled trial? </li></ul>
APPRAISING AN ARTICLE ON THERAPEUTIC INTERVENTIONS <ul><li>RELEVANCE QUESTION </li></ul><ul><li>1. Is the objective of the article comparing therapeutic interventions similar to your clinical dilemma? </li></ul><ul><li>a. population of the study </li></ul><ul><li>b. intervention & comparative interventions </li></ul><ul><li>c. outcome of the study </li></ul>
B. VALIDITY GUIDES 1. Was the assignment to the different treatment groups randomized? 2. Was follow-up complete? If with drop-outs , for results to be acceptable: a. Intention to Treat analysis b. < 20% of subjects c. explanation of why subjects dropped out 3. Was the study blinded? a. Single-blind b. Double-blind
4. Were the outcome measurements clearly described and determined in the same way between interventions? Can the study be replicated using the same methodology & measurements? Are these the outcomes you are aiming for? 5. Over-all, is the study valid?
C. WHAT ARE THE RESULTS? 1. What are the results? Risk in treatment (Rt) = # of pxs who did not get well in the treatment group/total # of pxs in the treatment group Risk in control (Rc) = # of pxs who did not get well in the control group/total # of pxs in the control group Relative Risk (RR) = Rt/Rc Relative Risk Reduction (RRR) = ( 1 – RR x 100%)
Absolute Risk Reduction (ARR) = Rc – Rt Number Needed to Treat (NNT) = 1/ARR Cost Effectiveness = NNT x dose x days x cost * RR = 1 : no difference between treatment & control RR > 1 : treatment is harmful RR < 1 : treatment is beneficial 2. Are the results reliable? Confidence Interval (CI) : > or = 95% p-value : < or = 0.05
D. APPLICABILITY GUIDES 1. Are the medical, social, & economic resources needed to administer the treatment available in your setting? 2. In your perception, were the treatment and its outcome as measured in the article preferred by the patient & his family? 3. Do you think that the patient, family, and/or community will be willing to accept and pay for the treatment in question? E. RESOLUTION OF THE THERAPEUTIC DILEMMA
CLINICAL SCENARIO <ul><li>35/M, employee </li></ul><ul><li>showed hematuria during a routine annual company PE </li></ul><ul><li>with known history of recurrent UTI in the past year </li></ul><ul><li>PWI: Urolithiasis </li></ul><ul><li>KUB-IVP or KUB-UTZ? </li></ul><ul><li>DILEMMA: Among patients with hematuria, </li></ul><ul><li>will a KUB-UTZ detect urolithiasis vs. KUB-IVP? </li></ul>
APPRAISING AN ARTICLE ON A DIAGNOSTIC TEST <ul><li>SCREENING QUESTION </li></ul><ul><li>Was the objective of the paper relevant to your clinical question? </li></ul><ul><li>VALIDITY </li></ul><ul><li>1. Was a representative sample of subjects to whom the diagnostic test is supposed to be administered gathered? </li></ul>
2. Was the diagnostic test and reference standard described in detail to permit reliable replication? Methodology must be detailed enough (e.g., patient preparation, ideal conditions to carry out test) for reliable replication 3. Was there an independent comparison with the reference standard? Verification bias: increases the accuracy of the test 4. Was the reading or interpretation of the diagnostic test or reference standard done independently?
RESULTS 1. What are the likelihood ratios for the different test results? DISEASE TEST
APPLICABILITY 1. Are the medical, social, & economic resources needed to perform the diagnostic test available in your setting? 2. Was the diagnostic test & accuracy in the evidence preferred by the patient & the family? 3. If the physician gives the diagnostic test to a similar patient in the community, will he get the same result to most of them?
4. Is the patient, family, and community willing to accept and pay for the diagnostic test? RESOLUTION OF THE SCENARIO