Considering adverse effects in prioritising reviews


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Considering adverse effects in prioritising reviews

    1. 1. Considering Adverse Effects in prioritising reviews Andrew Herxheimer Cochrane Adverse Effects Methods Group
    2. 2. Why include harmful effects in reviews? <ul><li>Considering only beneficial effects leads to bias </li></ul><ul><li>We must aim to assess benefit/ harm balance </li></ul><ul><li>Detailed evaluation needed esp. when: </li></ul><ul><ul><li>narrow margin between benefit and harm </li></ul></ul><ul><ul><li>(e.g. aspirin/ CVD) </li></ul></ul><ul><ul><li>effective treatments differ in their harm profiles </li></ul></ul><ul><ul><li>AEs make patients stop treatment </li></ul></ul>
    3. 3. Reconsidering the review questions <ul><li>What should the review questions be? </li></ul><ul><li>Depends on the aim of the review </li></ul><ul><ul><li>All adverse effects relevant to clinical decisions? </li></ul></ul><ul><ul><li>A specific treatment or a class of treatments? </li></ul></ul><ul><ul><li>Specific AEs or specific populations ? </li></ul></ul><ul><ul><li>AEs across all conditions treated or </li></ul></ul><ul><ul><li>use in one specific condition </li></ul></ul><ul><ul><li>All dosages ? </li></ul></ul><ul><ul><li>All durations of use? </li></ul></ul><ul><ul><li>With an evolving method, what cut-off point to use? </li></ul></ul>
    4. 4. Gaps & weaknesses in data <ul><li>poor or inaccessible data on human toxicology </li></ul><ul><li>patchy and unsatisfactory analysis of spontaneous reports of suspected ADRs </li></ul><ul><li>deficient monitoring of AEs in practice </li></ul><ul><li>no systematic study of mechanisms of AEs </li></ul><ul><li>little work on prevention or attenuation of AEs </li></ul><ul><li>no clear method to assess benefit/ harm balance </li></ul><ul><li>sparse knowledge of the social impact of medicines – on communities, development of dependence, quality of life … </li></ul>
    5. 5. Searching and Sources <ul><li>Need to search for studies of all designs, not just RCTs </li></ul><ul><li>AEs often treated as secondary outcomes </li></ul><ul><li>Poor reporting in titles, abstracts, so poor indexing </li></ul><ul><li>Inconsistent terminology and indexing </li></ul><ul><li>Many data sources: time and resource-intensive </li></ul><ul><li>Use industry, surveillance data, tertiary sources? </li></ul><ul><li>No consensus on best search strategy or sources </li></ul>
    6. 6. Which study designs? <ul><li>Most AE data come from studies other than RCTs - </li></ul><ul><li>Cohort studies, case-control studies, case series, case reports </li></ul><ul><li>So, new searches are needed </li></ul>
    7. 7. Quality assessment & evidence synthesis 1 <ul><li>The evidence hierarchy differs from that for evaluative research: RCTs are not on top – other study designs must also be considered </li></ul><ul><li>Each study design contributes a different category of evidence </li></ul><ul><li>These categories of evidence are combined to summarise quantitatively: a. the relation between the cause of a disease and the risk of the disease; b. the extent to which the disease can be prevented </li></ul><ul><li>Such a teleoanalysis , unlike a meta-analysis, combines data from different classes of evidence rather than from one type of study </li></ul><ul><li>See Wald NJ, Morris JK. BMJ 2003;327:616-8 </li></ul>
    8. 8. Quality assessment & evidence synthesis 2 <ul><li>Do we know how AEs were detected ? Is reporting adequate ? Under-reporting is universal! </li></ul><ul><li>Data are ‘thinner’ & less reliable than those for effectiveness </li></ul><ul><li>Needs more tolerance of ambiguity </li></ul><ul><li>The precautionary principle applies </li></ul>
    9. 9. Other difficulties <ul><li>Takes time & resources </li></ul><ul><li>Frequencies often can’t be estimated </li></ul><ul><li>It’s tempting to exclude evidence with high risk of bias , but it’s better to explain that </li></ul><ul><li>Are definitions/ grouping of outcomes comparable across studies? </li></ul><ul><li>An intervention may have many potential AEs which cannot be analysed in detail but only 2 or 3 beneficial outcomes </li></ul>
    10. 10. How all this influences priorities <ul><li>Some of a CRG’s existing reviews & protocols may need urgent revision to include AEs </li></ul><ul><li>Important or widely used interventions with ill understood AEs deserve priority </li></ul><ul><li>AEs of doubtfully or marginally effective but widely used interventions need prompt review </li></ul><ul><li>Members of the AEMG will be glad to help by discussing your specific examples </li></ul>
    11. 11. Working together <ul><li>Cochrane Adverse Effects Methods Group </li></ul><ul><li>Website ( </li></ul><ul><li>Advisory Team </li></ul><ul><li>Method Papers </li></ul><ul><li>Discussion List </li></ul><ul><li>Workshops </li></ul><ul><li>Enquiry Database ( FAQs ) </li></ul><ul><li>Contact person for each Review Group </li></ul><ul><li>Collecting examples of review decisions and how they were made </li></ul><ul><li>Ongoing Research </li></ul>
    12. 12. Guidelines so far <ul><li>Cochrane Handbook </li></ul><ul><ul><li>Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.0 [updated February 2008]. The Cochrane Collaboration, 2008. Available from </li></ul></ul><ul><li>BMC Paper </li></ul><ul><ul><li>Loke  YK, Price D, Herxheimer A. Systematic reviews of adverse effects: framework for a structured approach    BMC Med Res Methodol 2007;7:32. </li></ul></ul><ul><li>CRD Report 4 </li></ul><ul><ul><li>Forthcoming update/ intranet guidelines </li></ul></ul>