1.
Supporting authors of
systematic reviews.
Margaret Anderson
Trials Search Co-ordinator m.anderson@qub.ac.uk
Cochrane Developmental, Psychosocial and Learning Problems
Group
Queen’s University Belfast

2.
Systematic reviews in the social sciences
0
100
200
300
400
500
600
700
1
0
1 8
61
214
614
IBSS records
with "systematic review" in title or abstract.
No of IBSS records

3.
Systematic review
“A review of a clearly formulated question that uses
systematic and explicit methods to identify, select, and
critically appraise relevant research, and to collect and
analyse data from the studies that are included in the
review. Statistical methods (meta-analysis) may or may not
be used to analyse and summarise the results of the
included studies.”
Cochrane community website http://community.cochrane.org/glossary/5#letters accessed 28 January 2015

4.
Cochrane worldwide
• More than 22,400 active Cochrane authors worldwide
attached to one or more
• 53 Review Groups
• Publish protocols and reviews in Cochrane Database of
Systematic Reviews (> 8,500)
• Well defined subject scope
• Editorial base
• Typically a Co-editor, Managing Editor, Trials Search Co-
ordinator
• Editorial board
• Subject , methods and statistical expertise
• External peer reviewers

5.
Cochrane Developmental, Psychosocial
and Learning Problems Group
• Funded by HSC Northern Ireland
• Based at Queen’s University Belfast (since 2010)
• Editorial base
• Co-editor
• Managing Editor
• Assistant Managing Editor
• Trials Search Co-ordinator
• 992 authors
• 174 reviews
• 32 protocols
• 17 registered titles

6.
Review Life cycle (18 months to 2 years)
Title registered
and accepted by
Cochrane group
Write the
protocol
Feedback and
editorial approval
Publish the
protocol in CDSR
Run the
searches
Write the review
Feedback and
editorial approval
Publish the
review in CDSR
Update the
searches
Incorporate new
data
Feedback and
editorial approval
Publish review
update in CDSR

7.
Overview of review process: TSC input
• Review Proposal stage ( Title Registration)
• Protocol stage
• First draft – liaise with authors search
strategy
• Check subsequent drafts for revisions
• Run searches or check review teams progress
• Deliver records and summary of search
• Review stage – check all search related sections

8.
Overview of review process: TSC input
• Top-up searches
“Rerun or update searches for all relevant databases
within 12 months before publication of the review or
review update, and screen the results for potentially
eligible studies”*
C37 Methodological Standards for the Conduct of Cochrane Intervention
Reviews http://editorial-unit.cochrane.org/mecir

9.
Title registration
• Editorial discussion
• Check The Cochrane Library for similar
protocols or reviews
• Author team
• Who is developing the strategy and running searches?
• Search expertise?
• library access?
• advice from an information professional?

10.
Guidance and standards
• MECIR Project http://editorial-unit.cochrane.org/mecir
• Methodological standards for the conduct of new
Cochrane reviews
• Methodological standards for the reporting of new
Cochrane intervention reviews
• Cochrane Handbook for Systematic Reviews of
Interventions Version 5.1.0[updated March 2011]
• http://handbook.cochrane.org/
• Handbook for DTA Reviews (work in progress)
• http://srdta.cochrane.org/handbook-dta-reviews

11.
Protocol development
“Searches for studies should be as extensive as possible…” *
• Comprehensive list of sources (CDPLPG average is 12)
• Include mandatory and highly desirable sources**
• Published and unpublished sources
• Reference lists and personal contacts
yet also need to
• Avoid duplication
• Ensure list is realistic
• Access
• Review Team’s capacity
* C24 to C31 Methodological Standards for the Conduct of Cochrane Intervention Reviews
http://editorial-unit.cochrane.org/mecir

12.
Protocol: developing a search strategy
• Work from the protocol (not just the title)
• Is the intervention clear?
• What is the population (condition, age group)?
• Study types?
• RE-read the protocol!
• It might change after editorial feedback

13.
Protocol: developing a search strategy
• Engaging with the authors
• Try to provide encouraging feedback
• Ask them to suggest search terms (older
terms, regional terms)
• Check understanding (mine and authors)
• Seek clarification from authors
• Test drive the strategy
• Use references from other reviews
• Ask authors for likely eligible studies

14.
Features of the core search strategy
• published as part of the Protocol
• structured using main review concepts
• population AND intervention and RCT filter
• Controlled vocabulary
• Comprehensive free-text terms
• Balance sensitivity and precision
Item C32 Methodological Standards for the Conduct of Cochrane Intervention
Reviews http://editorial-unit.cochrane.org/mecir
Cochrane Handbook Sections 6.4.2; 6.4.4; 6.4.7

15.
Developing the core strategy
• Choose core database
• Sort out the structure
(Concept 1 AND Concept 2 AND Concept 3)
• Find controlled thesaurus terms
• Begin with the most obvious term and examine
broader and narrower terms .
• Work backwards from a known study
• Look at strategies from other reviews
• Test free text words and phrases one at a time
• Use a separate search line for each one
• What happens if …

16.
Preparing to run the searches
• Agree who is doing what and record the decision
• Discuss reference manager software
• Agree when to start (searches have a “shelf life”)**
• Agree who will receive the records and co-ordinate
screening
C37 Methodological Standards for the Conduct of Cochrane Intervention Reviews
http://editorial-unit.cochrane.org/mecir
Cochrane Handbook Sections 6.6.1

17.
Preparing to run the searches
• Customise the core strategy for other databases
• Check appropriate syntax and controlled
vocabulary
C33 Methodological Standards for the Conduct of Cochrane Intervention Reviews
http://editorial-unit.cochrane.org/mecir
Cochrane Handbook Sections 6.6.1

18.
Managing expectations: new CDPLPG reviews 2010-
2012
Maximum= 31879
Minimum =27
Number of reviews= 40

19.
Documenting the searches
• Record enough detail so search can be
replicated*
• Database name, provider and dates of
coverage
• Full name and URLs of websites
• Save exact search strategy for each source
• Collate search strategies as you go
*C36 Methodological Standards for the Conduct of Cochrane Intervention Reviews
http://editorial-unit.cochrane.org/mecir
Cochrane Handbook Sections 6.6.1

20.
Managing the records
• Downloading records
• Label downloaded records with database, search
date and number of records)
• Importing into reference manager
• Global edit to add database name
• Make separate RIS files for each source
• Keep a master copy of all records!
• Deduplication
• Keep deduplication settings conservative
• Move duplicates to a separate file
• Advise authors to expect more duplicates

21.
Reporting the searches
• Abstract (word limited)
• Search date
• the earliest date of the most recent set of searches
• must be within 12 months (and preferably 6
months) of the publication date
• List of databases
• Indicative -name the main ones then “and 5 others”)
R6 Methodological Standards for the reporting of new Cochrane Intervention Reviews
http://editorial-unit.cochrane.org/mecir

22.
Reporting the searches
• Search methods
• List all sources
• Include provider/dates of coverage
• Issue/version number (e.g. CENTRAL 2014
Issue 12)
• Full name and URL of websites
• Latest searches
• Date of latest searches
R34 R35 Methodological Standards for the Conduct of Cochrane
Intervention Reviews http://editorial-unit.cochrane.org/mecir

23.
PRISMA study flow diagram
Liberati A, Altman DG, Tetzlaff J, Mulrow C, et al. (2009) The PRISMA Statement for Reporting Systematic Reviews and Meta-
Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. PLoS Med 6(7): e1000100.
doi:10.1371/journal.pmed.1000100
http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.1000100

24.
Reporting the searches
• Study flow diagram (PRISMA)
• Ensure numbers add up
• Do numbers of records and studies tally?
• Check numbers of included and excluded studies
• Check studies used in any meta-analysis
• Appendix with search strategies
• One for each source

25.
Updates
• Generally expect to update every 2 years
• Monitor downloads
• Monitor citations (journals, policy documents clinical
guidelines (NICE, WHO)
• Be aware of plans to update guidelines
• Monitor Altmetrics*
• Update process begins with searches
• Chance to improve/update search strategy
*http://exchanges.wiley.com/authors/faqs---altmetrics_340.html

26.
Information specialist input
• Awareness of sources
• Advise on coverage
• Access
• Limitations
• Searching expertise
• Exploiting thesaurus terms
• Adapting searches to different databases
• Downloading and managing records
• Reporting
• Documenting searches
• Referencing
• Monitoring impact

27.
Other systematic reviews are available…
• Journals
• Not-for-profit
organisations
• Commercial
organisations
• Research
calls/commission
s