Systematic reviews of adverse effects and other topics not – yet – covered by the Cochrane Collaboration


Published on

Gerald Gartlehner speaking at the Symposium on the 10th Anniversary of the German Cochrane Centre

Published in: Health & Medicine, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Systematic reviews of adverse effects and other topics not – yet – covered by the Cochrane Collaboration

  1. 1. Systematic Reviews of Adverse Effects and Other Topics Not Yet Covered by Cochrane Reviews Gerald Gartlehner Donau-Universität Krems, Department für Evidenzbasierte Medizin und Klinische Epidemiologie Symposium anlässlich des 10-jährigen Bestehens des Deutschen Cochrane Zentrums 2. April 2008
  2. 2. Objectives <ul><li>To further the discussion on two important issues currently left incomplete by many Cochrane reports: </li></ul><ul><ul><ul><li>Adverse effects </li></ul></ul></ul><ul><ul><ul><li>Applicability of findings </li></ul></ul></ul>
  3. 3. Benefits Harms A systematic review emphasizing only benefits is likely to lead to biased conclusions
  4. 4. Cochrane Reviews Underemphasize Adverse Effects <ul><li>Of 138 Cochrane reviews with data from at least 4000 participants, only 18% included data on clearly defined adverse effects </li></ul><ul><li>(Papanikolaou et al. Am J Med 2004;117:582-589) </li></ul>
  5. 5. <ul><li>Most Cochrane reviews focus on data from RCTs </li></ul><ul><li>213 out of 5053 records mention “observational” OR “non-randomized” in the abstract </li></ul>
  6. 6. RCTs and Adverse Effects – Major Issues <ul><li>1. Assessment and reporting of harms is often inadequate </li></ul><ul><li>2. Methodological issues limit the detection of certain adverse effects </li></ul><ul><li>3. Publication bias </li></ul>
  7. 7. Inadequate Assessment and Reporting <ul><li>Review of 7 interventions (Ioannidis et al. JAMA 2001;285:437-443) </li></ul><ul><ul><li>61% had inadequate reporting of adverse effects and 71% of laboratory-determined toxicity </li></ul></ul><ul><ul><li>Space devoted to safety was the same or less than the space given for the names of authors and their affiliations </li></ul></ul><ul><li>Reporting may be worse for non-pharmacologic vs. pharmacologic interventions </li></ul><ul><li>(Ethgen et al. Ann Intern Med 2005;143:20-25) </li></ul>
  8. 8. Example: Second-generation Antidepressants <ul><li>Sexual adverse events are common but frequently underreported </li></ul><ul><li>Assessment in RCTs varied greatly </li></ul><ul><li>Most studies did not use targeted questions </li></ul>Gartlehner et al. Comparative Effectiveness Review No 7: Agency for Healthcare Research and Quality. January 2007
  9. 9. Reporting of Sexual Adverse Events in RCTs of Antidepressants
  10. 10. Mean Incidence of Sexual Adverse Events in RCTs
  11. 11. Mean Incidence of Sexual Adverse Events in Observational Studies
  12. 12. Methodological Limitations of RCTs to Assess Harms <ul><li>Sample size </li></ul><ul><li>Limited duration </li></ul><ul><li>Lack of applicability (homogeneous, healthy populations) </li></ul>
  13. 13. Methodological limitations of RCTs to assess adverse effects <ul><li>RCTs are a good primary source to assess harms, if adverse effects : </li></ul><ul><ul><li>are distinctive </li></ul></ul><ul><ul><li>occur frequently </li></ul></ul><ul><ul><li>have a close temporal association with the intervention </li></ul></ul><ul><li>RCTs provide an incomplete picture of the risk of harms if adverse effects are: </li></ul><ul><ul><li>infrequent </li></ul></ul><ul><ul><li>have a long latency </li></ul></ul><ul><ul><li>or affect only subgroups of patients </li></ul></ul>
  14. 14. Publication Bias: Antidepressants in Children <ul><li>Systematic review of published RCTs </li></ul><ul><li>Favorable risk benefit profiles for the treatment of depressive disorders in children </li></ul><ul><li>Inclusion of unpublished trials reversed conclusions for most antidepressants </li></ul>(Whittington et al. Lancet 2004; 363: 1341-45)
  15. 15. How can the limitations be overcome? <ul><li>Observational studies may provide useful supplemental information for assessing adverse effects </li></ul><ul><ul><li>Often larger than RCTs </li></ul></ul><ul><ul><li>Cover longer time periods </li></ul></ul><ul><ul><li>Examine less selected populations </li></ul></ul>
  16. 16. “ Observational studies” refers to a broad range of study designs <ul><li>Controlled observational studies (often derived from large databases) </li></ul><ul><ul><li>Cohort studies </li></ul></ul><ul><ul><li>Case-control studies </li></ul></ul><ul><li>Hypothesis-generating observational studies </li></ul><ul><ul><li>Case series, case reports </li></ul></ul><ul><ul><li>Data-mining </li></ul></ul>
  17. 17. Case reports <ul><li>30% of primary published literature on adverse drug events is in the form of case reports (Aronson et al.Fundam Clin Pharacol 2004;57:616-21) </li></ul><ul><ul><ul><li>Suicidality and SSRIs </li></ul></ul></ul><ul><ul><ul><li>Reye syndrome and aspirin </li></ul></ul></ul>
  18. 18. Observational Studies and Confounding <ul><li>Systematic error (unmeasured differences between groups) is always possible </li></ul><ul><li>Adverse effects are often unexpected and unpredictable </li></ul><ul><li>Confounding by indication is often not relevant </li></ul>
  19. 19. Asymptomatic Carotid Artery Surgery Trial (ACAS) <ul><ul><li>From Rothwell, Lancet 2005;365;82-93 </li></ul></ul>
  20. 20. Applicability - What Decision Makers Need To Know <ul><li>Can it work? </li></ul><ul><li>Will it work? </li></ul><ul><ul><li>In this patient? </li></ul></ul><ul><ul><li>In this setting? </li></ul></ul><ul><li>Is it worth it? </li></ul><ul><ul><li>Do benefits outweigh harms/costs/inconvenience? </li></ul></ul><ul><ul><li>Does it offer important advantages over existing alternatives? </li></ul></ul><ul><ul><ul><li>( adapted from Brian Haynes </li></ul></ul></ul><ul><ul><ul><li>ACP Journal Club) </li></ul></ul></ul>
  21. 21. APPLICABILITY ( external validity, generalizability) Can I apply results to my population of interest? Can it work (under ideal circumstances)? Will it work (in real life)?
  22. 22. How can we facilitate the assessment of applicability? <ul><li>Better reporting of factors that may affect applicability </li></ul><ul><ul><li>Population characteristics </li></ul></ul><ul><ul><li>Intervention and comparisons </li></ul></ul><ul><ul><li>Intended and Unintended Outcomes </li></ul></ul><ul><ul><li>Adverse effects </li></ul></ul><ul><ul><li>Differences between trial protocol and routine practice </li></ul></ul><ul><ul><li>(Rothwell, Lancet 2005;365;82-93) </li></ul></ul><ul><li>Standardized approach to distinguish efficacy from pragmatic studies </li></ul>
  23. 23. Criteria to distinguish efficacy from pragmatic trials <ul><li>Population in primary care </li></ul><ul><li>Less stringent eligibility criteria </li></ul><ul><li>Health outcomes </li></ul><ul><li>Clinically relevant study duration and treatment modalities </li></ul><ul><li>Assessment of adverse events </li></ul><ul><li>Adequate power to detect a minimally important difference from a patient perspective </li></ul><ul><li>ITT-analysis </li></ul>(Gartlehner et al. A simple and valid tool distinguished efficacy from effectiveness studies. J Clinical Epidemiology 2006; 56:1040-1048)
  24. 24. Summary <ul><li>Balanced systematic reviews should assess both benefits and harms </li></ul><ul><li>Broader range of data sources may be required </li></ul><ul><li>More emphasis on the applicability of findings </li></ul>