Critical appraisal of published medical research


Published on

Published in: Health & Medicine, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Critical appraisal of published medical research

  1. 1. Critical Appraisal of Published Medical Research Dr. Tarek Amin Professor of Public Health Cairo University
  2. 2. Background • Every day … – ~ 46 randomized clinical trials are published – ~ 1000 new Medline articles – ~ 6,000 new articles in biomedical journals • Every year … – ~ 3 million articles published in ~ 30,000 journals
  3. 3. Background • Most research published in medical journals is either – T poorly done oo – Insufficiently relevant to be clinically useful • Besieged with too much information to keep up to date. • High quality information is often not easy to find.
  4. 4. Critical appraisal is not o Negative dismissal of any piece of research o Assessment on results alone o Based entirely on statistical analysis o Undertaken by experts only
  5. 5. ?Why critically appraise • To find out the validity of the study – Are the methods robust? • To find out the reliability of the study – What are the results and are they credible? • To find out the applicability of the study – Is it important enough to change my practice?
  6. 6. ?What do I need to know o Awareness of study designs o Levels of evidence o Statistics!! o CA checklists o CA resources
  7. 7. Roadmap 1. 2. 3. 4. 5. Describe the evidence Internal validity External validity Comparison with other evidence Resources
  8. 8. Describe the evidence- 1  What relationship being evaluated and what hypothesis was tested?  What were the exposure and the outcome variable?  What was the study design?  Case report, series  Survey  Clinical trial  Case-control  Prospective or retrospective cohort study  Cross sectional study or  Others
  9. 9. Describe the evidence- 1  Definition of participants in terms of: – Source populations – Time frame – Eligibility criteria – Participation rates of the different groups compared  Summary of the main results:  What is the result in terms of association between exposure and outcome?  Should be possible to express the main result in a simple table and obtain from the paper the means to calculate the appropriate measure of association.
  10. 10. 2- Internal Validity o The truthfulness of inferences about the study population. o Causal relationship between exposure and outcome or just an association?
  11. 11. Internal Validity- 2 Two aspects of internal validity 1. Non-causal explanations 2. Causal explanations
  12. 12. Non-causal explanations • Bias • Confounding • Chance variation
  13. 13. Non-causal explanations • Bias – Selection bias, surveillance, diagnosis, referral, non-response, length of stay, survival bias – Misclassification bias: recall, interviewer, improper analysis, etc.,
  14. 14. Confounding Smoking is a risk factor for cancer of the larynx • we’ d like to quantify the strength of the association between smoking and laryngeal cancer, but … • many smokers are also drinkers (which is also a risk factor for cancer of the larynx) • drinking is said to confound the association between smoking and risk of laryngeal cancer
  15. 15. Confounding definition (Confounding (the formal definition • The effect of an extraneous variable that wholly or partially accounts for the apparent effect of the study exposure, or masks an underlying true association
  16. 16. A variable is confounder A variable is a confounder if: 1. It is causally associated with the outcome; and 2. It is non-causally associated with the exposure; and 3. It and the exposure variable are on two separate causal pathways
  17. 17. .Confounding A confounding variable is associated with the exposure and it affects the outcome, but it is not an intermediate link in the chain of causation between .exposure and outcome Oral contraceptive M yocardial infarction Smoking IUD insertion Salpingitis ST Ds
  18. 18. Chance variation A relationship between exposure and outcome identified by chance? Type I error: null hypothesis is rejected when, in reality, it is true.
  19. 19. Non-causal explanations The order of these non-causal explanations is :important o Observation (information) bias, analytical manipulation of the data will not overcome the problem o Confounding, then appropriate analysis will (in most cases) overcome the problem
  20. 20. Five aspects of causal explanations 1. 2. 3. 4. 5. Is there a correct temporal relationship? Is the relationship strong? Is there a dose-response relationship? Consistency of the association? Specificity of association
  21. 21. Is there a correct temporal . 1 ?relationship o The exposure must act before the outcome occurs o No problem with prospective study designs o Difficult in retrospective studies
  22. 22. ?Is the relationship strong. 2 Larger relative risks (and Odds) are more likely to reflect causal relationships.
  23. 23. Is there a dose-response. 3 ?relationship The greater the exposure, the greater the risk of disease.
  24. 24. Consistency of the association. 4 Expected to apply across a wide range of subjects. An association identified in one study that is consistent with the same association identified in a different groups of subjects.
  25. 25. Specificity of association. 5 Specificity: exposure produces a specific outcome (e.g. asbestos and mesothelioma)
  26. 26. External Validity- 3 External validity: can the results be applied to ?populations other than that which was studied • If the internal validity of a study is poor, the answer is no Aspects of external validity: 1. Applied to the eligible population? 2. Applied to the source population? 3. Applied to other relevant populations?
  27. 27. 1. Can the results be applied to the eligible population? – The relationship between the study population (the population from which samples are taken) and the eligible population (those that met the study inclusion criteria but did not take part) should be well documented. – Non-participation have to be considered carefully as they are likely to be non-random.
  28. 28. Can the results be applied to the. 2 ?source population Whether the association between outcome and exposure given by the study participants is likely to apply to other groups
  29. 29. Can the results be applied to other. 3 ?relevant populations The difficulties of applying results from one group of subjects to another will be minimal for issues of basic physiology and maximal for effects in which cultural and psycho-social aspects are dominant
  30. 30. Comparison with other evidence- 4 . Useful to consider a hierarchy of evidence 1. Randomized [clinical] trials 2. Cohort and case-control studies 3. Other comparative studies 4. Descriptive studies, case series, case studies, clinical experience
  31. 31. Three aspects of comparison should be :considered 1. Results consistent with other evidence? 2. Results plausible biologically? 3. Coherency with the existing knowledge.
  32. 32. ?Are the results consistent with other evidence • Most important characteristic used in the judgment that an association is causal • Lack of consistency argues against causality
  33. 33. Plausibility Is the observed association biologically understandable?
  34. 34. Coherency • An association is regarded as coherent if it fits the general features of the distribution of both the exposure and the outcome under assessment.
  35. 35. Summary 1. Describe the evidence • – type of study, outcome measure, population investigated, • results 2. Internal validity • – non-causal explanations • • bias • • confounding • • chance • – causal explanations • • temporal relationship • • strength of relationship • • dose-response • • consistency • • specificity
  36. 36. Summary 3. External validity • – can the results be applied to the eligible population? • – can the results be applied to the source population? • – can the results be applied to other relevant populations? 4. Comparison of the results with other evidence • – are the results consistent with other evidence? • – are the results plausible biologically? • – is there coherency with the distribution of the exposure and the • outcome? • Can we apply these results to other populations? Are the findings reported here consistent with other studies that looked at the same thing?
  37. 37. Resources Web sites – Users’ Guides to Evidence-Based Practice – A Student’ s Guide to the Medical Literature – Pearls for Residents: Annotated Critical Appraisal References %20residents/default.asp?s=1
  38. 38. • Critical Appraisal of Bio-medical Literature s.htm • Critical Appraisal Resources for Assessing Health and Medical Research ut.pdf • Bandolier
  39. 39. Thank you