Centre for Reviews and Dissemination
Systematic reviews: Searching for
information on adverse effects
Dr Su Golder su.gold...
Centre for Reviews and Dissemination
Systematic reviews: Searching for
information on adverse effects
Dr Su Golder
su.gold...
Centre for Reviews and Dissemination
Structure for Today’s Webinar
Why we should search for information on
adverse effect...
Centre for Reviews and Dissemination
Why search for information on
adverse effects?
Unpleasant, often serious – hospitali...
Centre for Reviews and Dissemination
Why is information on adverse
effects difficult to identify?
• AEs often treated as s...
Centre for Reviews and Dissemination
Where do authors of systematic
reviews search for adverse effects?
Analysis of 849 r...
Centre for Reviews and Dissemination
Quiz Time
Q: Which are the top three resources used
to identify information on advers...
Centre for Reviews and Dissemination
Which are the top three resources
used to identify information on
adverse effects?
1....
Centre for Reviews and Dissemination
How do authors of systematic
reviews search for adverse effects?
62% search with adv...
Centre for Reviews and Dissemination
The evidence on where to search for
adverse effects
Systematic review comparing sour...
Centre for Reviews and Dissemination
Systematic review of previous
research (Golder et al 2010)
 Objective
Summarise all...
Centre for Reviews and Dissemination
Case study with a drug intervention
(Golder et al 2012a)
Long-term use of glitazones ...
Centre for Reviews and Dissemination
Quiz Time
Q: Which database retrieved the highest
number of relevant records for this...
Centre for Reviews and Dissemination
Case study with a drug intervention: top
databases (Golder et al 2012a)
0%
10%
20%
30...
Centre for Reviews and Dissemination
Case study with a drug intervention:
unique records (Golder et al 2012a)
0
2
4
6
8
10...
Centre for Reviews and Dissemination
Case study with a drug intervention:
sources required (Golder et al 2012a)
Minimum co...
Centre for Reviews and Dissemination
Case study with a medical device
(unpublished)
Safety of recombinant human bone
morph...
Centre for Reviews and Dissemination
Case study with a medical device: top
databases
0%
10%
20%
30%
40%
50%
60%
70%
Embase...
Centre for Reviews and Dissemination
Case study with a medical device: unique
records
0
1
2
3
4
5
6
7
8
9
Science Citation...
Centre for Reviews and Dissemination
Case study with a medical device:
sources required
Minimum combination of sources
Sc...
Centre for Reviews and Dissemination
The evidence on how to search for
adverse effects
Analysis of records from 3 systema...
Centre for Reviews and Dissemination
Analysis of records from 3 systematic
reviews of adverse drug effects (Derry
et al 20...
Centre for Reviews and Dissemination
Analysis of records from 26
systematic reviews of adverse drug
effects (Golder et al ...
Centre for Reviews and Dissemination
Database records with any adverse
effects terms (Golder et al 2012b)
0%
10%
20%
30%
4...
Centre for Reviews and Dissemination
Quiz Time
Q: Which of the following search terms
retrieves the highest number of rele...
Centre for Reviews and Dissemination
Top search terms in MEDLINE
(Golder et al 2012b)
1 ‘adverse effects (ae)’ Floating su...
Centre for Reviews and Dissemination
Top search terms in Embase
(Golder et al 2012b)
1 ‘adverse drug reaction(ae)’ Floatin...
Centre for Reviews and Dissemination
Analysis of records from 26
systematic reviews of adverse drug
effects (Golder et al ...
Centre for Reviews and Dissemination
Average sensitivity of adverse effects
search filters in 26 systematic reviews
(Golde...
Centre for Reviews and Dissemination
Analysis of records from case study
systematic review of a medical
device (unpublishe...
Centre for Reviews and Dissemination
Top search terms for adverse effects
of a medical device in MEDLINE
1 ‘adverse effect...
Centre for Reviews and Dissemination
Top search terms for adverse effects
of a medical device in Embase
1 ‘complication (c...
Centre for Reviews and Dissemination
Take home messages
Including adverse effects in systematic reviews is
important so t...
Centre for Reviews and Dissemination
Take home messages
Adverse effects terms increasingly prevalent in title,
abstract o...
Centre for Reviews and Dissemination
Future
More reviews are including adverse effects
either as secondary outcome (in ad...
Centre for Reviews and Dissemination
Guidance
 Cochrane Handbook
Higgins JPT, Green S (editors). Cochrane Handbook for
Sy...
Centre for Reviews and Dissemination
Help and support
 Cochrane Adverse Effects Methods Group
http://aemg.cochrane.org/
...
Centre for Reviews and Dissemination
Any questions?
Centre for Reviews and Dissemination
References
Badgett R, Chiquette E, Anagnostelis B, Mulrow C. Locating reports of seri...
Centre for Reviews and Dissemination
References
Golder S, Loke YK. The contribution of different information sources for a...
Upcoming SlideShare
Loading in …5
×

Systematic Reviews: Searching for information on adverse effects by Dr. Su Golder, University of York, UK

1,163 views
884 views

Published on

In this webinar, Dr. Su Golder, an information specialist at the University of York and with 15 years experience with systematic reviews, covered the following:
- Why we should search for information on adverse effects?
- Why information on adverse effects difficult to search for?
- How we currently search for information on adverse effects?
- How we should search for information on adverse effects?

Published in: Health & Medicine, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,163
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
36
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

  • This tutorial will give guidance on the relative value and contribution of searching different sources

    (database and non-database) to identify adverse effects data. The tutorial will use a scenario drawn

    from a real-life situation, namely a systematic review and meta-analysis of thiazolidinedionerelated fractures in patients with type 2 diabetes mellitus. A detailed evaluation of each source

    searched will be presented including the sensitivity, precision, and number needed to read from

    searching each source and from different combinations of sources, and the effect of searching the

    sources in different orders. At the end tips and tricks to solve specific issues in identifying information

    on adverse effects will be given.

  • This tutorial will give guidance on the relative value and contribution of searching different sources

    (database and non-database) to identify adverse effects data. The tutorial will use a scenario drawn

    from a real-life situation, namely a systematic review and meta-analysis of thiazolidinedionerelated fractures in patients with type 2 diabetes mellitus. A detailed evaluation of each source

    searched will be presented including the sensitivity, precision, and number needed to read from

    searching each source and from different combinations of sources, and the effect of searching the

    sources in different orders. At the end tips and tricks to solve specific issues in identifying information

    on adverse effects will be given.
  • Our core work programme is funded by the NIHR, England; the Department of Health, Public Health Agency, Northern Ireland; and the National Institute for Social Care and Health Research, Welsh Assembly Government.
  • Adverse effects subgroup??
  • Systematic Reviews: Searching for information on adverse effects by Dr. Su Golder, University of York, UK

    1. 1. Centre for Reviews and Dissemination Systematic reviews: Searching for information on adverse effects Dr Su Golder su.golder@york.ac.uk
    2. 2. Centre for Reviews and Dissemination Systematic reviews: Searching for information on adverse effects Dr Su Golder su.golder@york.ac.uk
    3. 3. Centre for Reviews and Dissemination Structure for Today’s Webinar Why we should search for information on adverse effects? Why information on adverse effects difficult to search for? How we currently search for information on adverse effects? How we should search for information on adverse effects?
    4. 4. Centre for Reviews and Dissemination Why search for information on adverse effects? Unpleasant, often serious – hospitalisation, disability, death (USA: 4th to 6th leading cause of death) (Lazarou 1998) Worsen quality of life, make people stop treatment Cost (estimates of £2 billion per year to UK NHS) (Compass 2008) Can be a deciding factor in decision-making
    5. 5. Centre for Reviews and Dissemination Why is information on adverse effects difficult to identify? • AEs often treated as secondary or even tertiary outcomes. Poor reporting or absence of adverse effects terms in titles, abstracts and indexing • Wide range of terms for adverse effects. Inconsistent terminology and indexing • False hits; ‘Relative Risk’, ‘Self- harm’, ‘Patient safety’, ‘adverse effects were not considered’ • May wish to identify all adverse effects. Hard to predict/plan (specific terms may not be known in advance) • Relevant adverse effects may come from a range of study designs, not just RCTs • Adverse effects may not be limited to a particular condition • Many data sources: specialist databases, unpublished data, industry funded data, surveillance data, tertiary sources
    6. 6. Centre for Reviews and Dissemination Where do authors of systematic reviews search for adverse effects? Analysis of 849 reviews of adverse effects (Golder et al 2013, Golder et al 2014) Number of databases searched  median 2 (range 0 to 25)  increasing over time  greater if information professional involved
    7. 7. Centre for Reviews and Dissemination Quiz Time Q: Which are the top three resources used to identify information on adverse effects? A: Cochrane Library, MEDLINE and contacting experts B: MEDLINE, reference checking, and Embase C: Embase, Derwent Drug File and BIOSIS Previews
    8. 8. Centre for Reviews and Dissemination Which are the top three resources used to identify information on adverse effects? 1. MEDLINE: 96% 2. Reference checking: 76% 3. Embase: 54% 4. CENTRAL or Cochrane Library: 45% 5. Contacting experts: 22%
    9. 9. Centre for Reviews and Dissemination How do authors of systematic reviews search for adverse effects? 62% search with adverse effects terms 46% with named adverse effect e.g. headache, bleed, sepsis 5% with generic adverse effect terms e.g. adverse effects, side effects, complications etc. 11% with both
    10. 10. Centre for Reviews and Dissemination The evidence on where to search for adverse effects Systematic review comparing sources of information on adverse effects (Golder et al 2010) Case study systematic review of glitazones and fractures (Golder et al 2012a) Case study systematic review of the safety of spinal fusion (unpublished)
    11. 11. Centre for Reviews and Dissemination Systematic review of previous research (Golder et al 2010)  Objective Summarise all the literature comparing 2 or more sources to identify adverse effects  Results 19 included studies 8/10 cases searching Embase retrieved more relevant references than MEDLINE  Limitations Many of the included studies are out of date Little overlap in the sources compared
    12. 12. Centre for Reviews and Dissemination Case study with a drug intervention (Golder et al 2012a) Long-term use of glitazones and fractures in type 2 diabetes  Searched over 60 sources (beyond usual practice)  Used intervention (glitazones) and outcome (fractures) search terms  No diabetes terms used  Multiple textwords and indexing
    13. 13. Centre for Reviews and Dissemination Quiz Time Q: Which database retrieved the highest number of relevant records for this review on fracture and glitazones? A: MEDLINE B: Embase C: Science Citation Index (SCI)
    14. 14. Centre for Reviews and Dissemination Case study with a drug intervention: top databases (Golder et al 2012a) 0% 10% 20% 30% 40% 50% 60% 70% Percentage of all publications retrieved (n=58)
    15. 15. Centre for Reviews and Dissemination Case study with a drug intervention: unique records (Golder et al 2012a) 0 2 4 6 8 10 12 GSK Registry Science Citation Index (SCI) Embase BIOSIS Previews British Library Direct Medscape DrugInfo Unique relevant records
    16. 16. Centre for Reviews and Dissemination Case study with a drug intervention: sources required (Golder et al 2012a) Minimum combination of sources Science Citation Index Embase BIOSIS Previews GSK website Medscape DrugInfo British Library Direct Thomson Reuters Integrity* Conference Papers Index* AHFS First Handsearching** Reference checking *either database ** ten key journals
    17. 17. Centre for Reviews and Dissemination Case study with a medical device (unpublished) Safety of recombinant human bone morphogenetic protein-2 (rhBMP-2) Searched 10 databases plus reference checking, contacting authors and automated current awareness service Used intervention terms; recombinant human bone morphogenetic protein-2 (rhBMP-2) and spinal fusion Multiple textwords and indexing
    18. 18. Centre for Reviews and Dissemination Case study with a medical device: top databases 0% 10% 20% 30% 40% 50% 60% 70% Embase Science Citation Index (SCI) MEDLINE PubMED CENTRAL ToxFile Percentage of all publications retrieved (n=82)
    19. 19. Centre for Reviews and Dissemination Case study with a medical device: unique records 0 1 2 3 4 5 6 7 8 9 Science Citation Index (SCI) Embase CENTRAL MEDLINE PubMED ToxFile Unique relevant records
    20. 20. Centre for Reviews and Dissemination Case study with a medical device: sources required Minimum combination of sources Science Citation Index (SCI) Embase CENTRAL MEDLINE or PubMED Reference checking Contacting authors Automated current awareness service
    21. 21. Centre for Reviews and Dissemination The evidence on how to search for adverse effects Analysis of records from 3 systematic reviews of drug interventions (Derry et al 2001) Analysis of records from 26 systematic reviews of drug interventions (Golder et al 2012b) Analysis of records from case study systematic review of a medical device (unpublished)
    22. 22. Centre for Reviews and Dissemination Analysis of records from 3 systematic reviews of adverse drug effects (Derry et al 2001)  Objective To assess the presence or absence of adverse effects terms in the title, abstract or indexing of records of articles with adverse effects data  Results 23% of trials that reported adverse effects data had no adverse effects terms in title, abstract or indexing of records in MEDLINE or Embase  Guidance Do not rely on adverse effects terms Check full-text versions of retrieved articles
    23. 23. Centre for Reviews and Dissemination Analysis of records from 26 systematic reviews of adverse drug effects (Golder et al 2012b) Objective 1 To assess the presence or absence of adverse effects terms in the title, abstract or indexing of records of articles with adverse effects data published since 2001 Results 8% of articles published after 2001 that reported adverse effects data had no adverse effects terms in title, abstract or indexing of records in MEDLINE or Embase
    24. 24. Centre for Reviews and Dissemination Database records with any adverse effects terms (Golder et al 2012b) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Terms in any location Terms in title or abstract Indexing terms MEDLINE EMBASE Science Citation Index (SCI)
    25. 25. Centre for Reviews and Dissemination Quiz Time Q: Which of the following search terms retrieves the highest number of relevant records in MEDLINE? A: ‘adverse effects’ as a subheading B: ‘adverse adj3 event$’ in title or abstract C: ‘safety’ in title or abstract
    26. 26. Centre for Reviews and Dissemination Top search terms in MEDLINE (Golder et al 2012b) 1 ‘adverse effects (ae)’ Floating subheading 51% 2 ‘adverse adj3 event$’ Title or abstract 32% 3 ‘safety’ Title or abstract 31% 4 ‘adverse adj2 events’ Title or abstract 29% 5 ‘risk’ Title or abstract 28% 6 ‘drug effects (de)’ Floating subheading 27% 7 ‘complications (co)’ Floating subheading 18%
    27. 27. Centre for Reviews and Dissemination Top search terms in Embase (Golder et al 2012b) 1 ‘adverse drug reaction(ae)’ Floating subheading 83% 2 ‘side effect(si)’ Floating subheading 83% 3 exp drug safety/ Emtree indexing term 38% 4 ‘adverse adj3 event$’ Title or abstract 32% 5 ‘safety’ Title or abstract 28% 6 ‘adverse adj2 events’ Title or abstract 28% 7 ‘risk’ Title or abstract 27%
    28. 28. Centre for Reviews and Dissemination Analysis of records from 26 systematic reviews of adverse drug effects (Golder et al 2012c) Objective 2 The second objective of this analysis was to measure the performance of suggested adverse effects search filters/hedges
    29. 29. Centre for Reviews and Dissemination Average sensitivity of adverse effects search filters in 26 systematic reviews (Golder et al 2012c) 0% 20% 40% 60% 80% 100% 120% Buckingham with quality filter Buckingham without quality filter BMJ Clinical Evidence Badgett Golder excluding specific adverse effects Golder including specific adverse effects MEDLINE EMBASE
    30. 30. Centre for Reviews and Dissemination Analysis of records from case study systematic review of a medical device (unpublished)  Objective To assess the presence or absence of adverse effects terms in the title, abstract or indexing of records of articles with adverse effects data  Results  4% of articles that reported adverse effects data of a medical device had no adverse effects terms in title, abstract or indexing of records in MEDLINE or Embase
    31. 31. Centre for Reviews and Dissemination Top search terms for adverse effects of a medical device in MEDLINE 1 ‘adverse effects (ae)’ Floating subheading 47% 2 ‘complication$’ Title or abstract 35% 3 ‘postoperative complications/’ MeSH indexing term 27% 4 ‘safety’ Title or abstract 20% 5 ‘safely’ Title or abstract 20% 6 ‘blood loss’ Title or abstract 20%
    32. 32. Centre for Reviews and Dissemination Top search terms for adverse effects of a medical device in Embase 1 ‘complication (co)’ Floating subheading 49% 2 ‘complication$’ Title or abstract 35% 3 ‘pseudarthrosis/’ Emtree indexing term 24% 4 ‘adverse drug reaction (ae)’ Floating subheading 22% 5 ‘postoperative complication/’ Emtree indexing term 20% 6 ‘blood loss’ Title or abstract 18% 7 ‘bleeding/’ Emtree indexing term 18% 8 ‘dysphagia/’ Emtree indexing term 18%
    33. 33. Centre for Reviews and Dissemination Take home messages Including adverse effects in systematic reviews is important so that clinicians, patients and policy makers can make balanced decisions and minimise harm. Searches of multiple databases and non-database sources are required in systematic reviews of adverse effects. Searching only MEDLINE may miss over 40% of the relevant references.
    34. 34. Centre for Reviews and Dissemination Take home messages Adverse effects terms increasingly prevalent in title, abstract or indexing Searchers may cautiously rely on adverse effects search terms Indexing terms for adverse effects much more prevalent in Embase Subheadings particularly useful in Embase and MEDLINE
    35. 35. Centre for Reviews and Dissemination Future More reviews are including adverse effects either as secondary outcome (in addition to effectiveness) or as primary outcome Better reporting CONSORT Extension for Harms (Ioannidis et al 2004) PRISMA Harms Extension (Zorzela et al 2014)
    36. 36. Centre for Reviews and Dissemination Guidance  Cochrane Handbook Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.  CRD’s Guidance Systematic Reviews: CRD’s guidance for undertaking reviews in health care. http://www.york.ac.uk/inst/crd/pdf/Systematic_Reviews.pdf  BMC Paper Loke YK, Price D, Herxheimer A. Systematic reviews of adverse effects: framework for a structured approach. BMC Med Res Methodol 2007;7:32.
    37. 37. Centre for Reviews and Dissemination Help and support  Cochrane Adverse Effects Methods Group http://aemg.cochrane.org/  Discussion List http://lists.cochrane.org/mailman/listinfo/aemg  Twitter @CAEMG1
    38. 38. Centre for Reviews and Dissemination Any questions?
    39. 39. Centre for Reviews and Dissemination References Badgett R, Chiquette E, Anagnostelis B, Mulrow C. Locating reports of serious adverse drug reactions (PowerPoint presentation).[1999. http://medinformatics.uthscsa.edu/#FILTERS Buckingham J., Fisher B., Saunders D. Evidence based medicine tool kit: search strategies for articles on harm/etiology [Internet] Alberta, AB, Canada: University of Alberta; 2005 BMJ Clinical Evidence. Search filters. BMJ [Internet] 2006 [cited 9 May 2006]. http://www.clinicalevidence.com/ceweb/about/search_filters.jsp. Chou R, Aronsonb N, Atkinsc D, Ismailad AS, Santaguidad P, Smith DH, et al. AHRQ Series Paper 4: Assessing harms when comparing medical interventions: AHRQ and the Effective Health-Care Program J Clin Epidemiol 2010;63:502-12. Compass. Adverse drug reactions wastes NHS £2BN reveals Compass. Compass; London; 2008. [cited 2008 18 Sep]. Available from: http://www.compassonline.org.uk/news/item.asp?n=1551. Derry S, Loke YK, Aronson JK. Incomplete evidence: the inadequacy of databases in tracing published adverse drug reactions in clinical trials. BMC Med Res Methodol 2001;1:7. Golder S, McIntosh HM, Duffy S, Glanville J, Developing efficient search strategies to identify reports of adverse effects in MEDLINE and EMBASE. Health Info Libr J. 2006 Mar;23(1):3-12. Golder S, Loke YK. Sources of information on adverse effects: a systematic review. Health Info Libr J 2010;27(3)176-90.
    40. 40. Centre for Reviews and Dissemination References Golder S, Loke YK. The contribution of different information sources for adverse effects data. Int J Technol Assess Health Care. 2012a;28(2):133-7. Golder S, Loke YK. Failure or success of electronic search strategies to identify adverse effects data. JMLA 2012b;100(2):130-4. Golder S, Loke YK. The performance of adverse effects search filters in MEDLINE and EMBASE. Health Info Libr J 2012c;29(2):141-51. Golder S, Loke YK, Zorzela L. Some improvements are apparent in identifying adverse effects in systematic reviews from 1994 to 2011. J Clin Epidemiol 2013;66(3):253-60. Golder S, Loke YK, Zorzela L. Comparison of search strategies in systematic reviews of adverse effects to other systematic reviews. Health Info Libr J 2014. Ioannidis JP, Evans SJ, Gotzsche PC, O'Neill RT, Altman DG, Schulz K, Moher D. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med 2004; 141(10):781-788. Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reaction in hospitalized patients: a meta-analysis of prospective studies. JAMA 1998;279:1200-5. Wieland S, Dickersin K. Selective exposure reporting and Medline indexing limited the search sensitivity for observational studies of the adverse effects of oral contraceptives. J Clin Epidemiol. 2005 Jun;58:6:560-7. Zorzela L, Golder S, Liu, Y, Pilkington K, Hartling L, Joffe A, Loke Y, Vohra S. Quality of reporting in systematic reviews of adverse events: systematic review. BMJ. 2014;348.

    ×