Literature Searching
for Systematic Reviews
January 2009
Diane Lorenzetti MLS
Centre for Health & Policy Studies
Institute of Health Economics
dllorenz@ucalgary.ca

A systematic review is..
“a review of the evidence on a clearly formulated
question that uses systematic and explicit methods
to identify, select and critically appraise relevant
primary research, and to extract and analyse data
from the studies that are included in the review
Statistical methods (meta-analysis) may or may not
be used.”
(Undertaking systematic reviews of research on effectiveness: CRD's guidance for
those carrying out or commissioning reviews (2001)
http://www.york.ac.uk/inst/crd/report4.htm.)

Characteristics of
systematic literature searches
Meticulous planning
•
Comprehensive method for searching the literature
•
Inclusion of a variety of sources of information
•
Identification of all relevant studies
•
Unbiased searching (eg: inclusion of non-english language
•
research and both published and unpublished studies)
Transparent and reproducible search strategy
•

Before beginning a systematic review…
Establish the need for a systematic review
Document decisions made by the team during the
question refinement and literature search process
Develop a timeline for completion of each stage of your
review

A timeline may include:
Pre-searching and question refinement
•
Development of inclusion/exclusion criteria
•
Development of search strategy
•
Literature searching (including searching of the grey
•
literature and strategy refinement)
Development of abstract inclusion/exclusion criteria
•
Abstract review and selection
•
Document retrieval
•
Creation of data extraction form for full text article review
•
Full text article review and selection
•
Synthesis
•
Preparation of the final report
•
Many of these activities may occur simultaneously……………

Pre-searching and Question Refinement
Develop a clear question
Conduct a pre-search prior to beginning your review
Be prepared to revise your question based on the results
of your pre-search

PICO
PICO is a model adopted by proponents of evidence-based
medicine as an aid to question formulation

Developing a research question using
PICOS
P = Population: the group under observation/study
I = Intervention: the intervention, exposure or action
of interest
C = Comparison an alternative action or intervention
against which the intervention of
interest will be compared
O = Outcome: a measure of the desired effect of
the Intervention/Exposure
S = Study design study designs appropriate for the
research question under consideration

Components of a research question
(PICO format)
Q1: Is a pre-recorded telephone health information service
(Intervention) for mothers with toddlers (Population)
more effective than printed information (Comparison) in
increasing knowledge of accident prevention
techniques? (Outcome)
Q2: Is there a relationship between health care costs
(Intervention) and patients’ (Population) use of health
services (Outcome)?

Keywords & synonyms
Develop lists of keywords and synonyms for each
PICO component of your research question
before you begin searching for relevant literature.
Keywords are independent of any one
electronic database and will help you to be
consistent as you map your search to each
database/resource you use.

Keywords & synonyms
Eg: Is hypnotism an effective means of reducing
smoking in adolescents?
Concept/ Teenagers Hypnotism Smoking
Facet P I O
Keywords/ teenager(s) hypnotism smoke
Synonyms teen(s) hypnosis smoking
adolescent(s) hypnotherapy cigarette(s)
youth(s) suggestion tobacco
high school snuff
student(s)

Boolean operators: OR
Used to connect synonyms, similar terms
Broadens / expands a search
Finds records with any of your search terms
Eg: glucosamine OR n-acetyl-d-glucosamine

Boolean operators: AND
Used to combine search concepts/components
Narrows / focuses a search
Finds records with all your search terms
Eg: osteoarthritis AND glucosamine

Is hypnotism an effective means of reducing
smoking in adolescents?
PICO Teenagers Hypnotism Smoking
Component P I O
AND AND
Keywords/ teenager(s) hypnotism smoke
OR OR OR
Synonyms
teen(s) hypnosis smoking
OR OR OR
adolescent(s) hypnotherapy cigarette(s)
OR OR OR
high school suggestion tobacco
student(s) OR
snuff

Sensitivity and specificity in literature searching
Sensitivity (recall)
The proportion of relevant articles identified by a search strategy as a
percentage of all relevant articles on a given topic. It is a measure of
the ability of a search strategy to identify all relevant articles.
Specificity (precision)
The proportion of relevant articles identified by a search strategy as a
percentage of all articles (relevant and irrelevant) identified by that
search. It is a measure of the ability of a search strategy to exclude
irrelevant articles.
Highly sensitive strategies tend to have low levels of precision and vice
versa. In a systematic review, the aim is to maximize sensitivity while also
attempting, insofar as it is possible, to maximize precision.

Study design filters
“a predefined search strategy designed to retrieve levels of
evidence (RCTs, systematic reviews etc) or types of clinical
queries (diagnosis, prognosis, etiology, treatment) when
combined with the subject search terms of your choice.
They are also referred to as hedges, Clinical Queries (USA),
or optimal search strategies.” (BestBETS Search Strategies
http://www.bestbets.org/links/strategies.html)
Used to limit search results by study design
Study design filters can increase the specificity of search
results

Study design terms found in select filters
Qualitative research
(e.g. “findings”, “qualitative”, “interview$”)
Diagnosis
(e.g. “sensitivity”, “specificity”)
Prognosis
(e.g. “follow-up studies”, “incidence”)
Intervention/Therapy
(e.g. “randomised controlled trial.pt.”)
Causation
(e.g. “risk”, “risk factor” and “cohort studies”)

RCT Sensitive filter for OVID MEDLINE searches
(Cochrane Collaboration)
(controlled clinical trial or randomized controlled trial or
1.
meta analysis).pt.
clinical trials as topic/
2.
(placebo$ or random$ or trial$).ti,ab.
3.
1 or 2 or 3
4.
limit 4 to animals
5.
limit 4 to (animals and humans)
6.
5 not 6
7.
4 not 7
8.

RCT Sensitive filter for OVID EMBASE searches
(Cochrane Collaboration)
cross-over procedure/ or double-blind procedure/ or
1.
randomized controlled trial/ or single-blind procedure/
(allocat$ or assign$ or cross over$ or crossover$ or
2.
(double ADJ blind$) or factorial or placebo$ or random$ or
(single ADJ blind$) or volunteer$).ti,ab.
1 or 2
3.
limit 3 to animals
4.
limit 3 to (animals and humans)
5.
4 not 5
6.
3 not 6
7.

Bibliographic database selection
No one bibliographic database is comprehensive enough to index all
published literature on a topic. MEDLINE indexes approx 5000 of the
20,000 biomedical journals published.
Overlap between databases is common
Extent of overlap between MEDLINE and EMBASE is estimated to be
34%; ranging between 10% and 75%, depending on topic
Selection will vary with the subject. When choosing electronic
databases, consider the multidisciplinary needs of your research
question & select databases that reflect these needs

Bibliographic databases for
clinical research topics
MEDLINE/PubMed

EMBASE

Cochrane Database of Systematic Reviews

Cochrane CENTRAL Register of Controlled Trials

CINAHL

PsycINFO

International Pharmaceutical Abstracts


Bibliographic databases for non clinical
research topics
ABI Inform

health policy, health services administration, economics of health care

ERIC

health promotion, health education

Social Sciences Abstracts & Sociological Abstracts

health policy, health promotion, patient access to and satisfaction with health care

Econlit

economics of health care

Philosophers’ Index

Ethical aspects of health care procedures and delivery


Database Thesauri/Subject Lists
Search both general and specific thesaurus/subject terms
•
as appropriate
MEDLINE examples:
Mental health[MeSH] plus MeSH terms for specific
mental health conditions such as schizophrenia[MeSH],
alcoholism[MeSH] etc…. as appropriate
Angiotensin-converting enzyme inhibitors[MeSH]
(ACE inhibitors used to treat high blood pressure and
heart failure) plus captopril[MeSH] (a specific ACE
inhibiting drug)

Broadening a search
Use broader, more general search terms
Include a variety of synonyms
Apply truncation symbols to word stems to retrieve
alternate spellings, singular/plural word forms &
word variants
Drop the least important concepts from your
search – eg: outcomes
Apply fewer / no limits to your search

Truncation
Finds variations of a word stem
Eg: plan$ retrieves plan, plans, planner, planners,
and planning
Truncation should only be used with keyword or
title/abstract word searching - not with subject
heading searching
Truncation symbol varies with database -
usually * but OVID databases use $

Narrowing a search
Use more specific search terms
Decrease # of synonyms included in the search
Eliminate or limit word truncation
Add concepts to your search
Apply limits

Searching Norms for Systematic Reviews
Search both thesaurus/subject terms and text
•
(title/abstract) words
Include both synonyms and truncated forms of text words
•
in your search
Include CAS Registry numbers and generic/brand names
•
in drug searches
Unless there is a compelling reason (eg: changes in
•
policies, procedures, interventions etc…) do not limit your
search by date or language

Pre-testing Your Search Strategy
Select one electronic database on which to craft, test and
finalize your search strategy
Be prepared to run multiple tests of your search strategy
with your research team before finalizing your approach
If you have pre-identified key studies, these studies can
act as a litmus test of the effectiveness of your search
strategy

Going beyond Electronic Databases
Consider searching beyond electronic databases to locate
•
relevant studies:
a) Reference lists of key articles
b) Hand searching of key journals
c) Hand searching of key conference proceedings
d) Consultation with experts
e) Citation searching with Web of Science
f) Grey literature

Grey literature may include…..
Unpublished or ongoing studies

Conference and meeting proceedings and abstracts

Dissertations & theses

Newsletters

Reports of research organizations and government

agencies
Informal communications

http://library.ucalgary.ca/branches/hsl/greylit

Why hand search journals?
Key journals may not be indexed in mainstream electronic

databases
Examples: complementary and alternative medicine & health

promotion
Inaccurate indexing of studies in electronic databases

Incomplete search strategies may result in studies being

missed during database searching
Journals indexed in databases such as MEDLINE and

EMBASE may be indexed selectively
Abstracts, letters or conference reports may not be indexed

in electronic databases

Cochrane Collaboration
hand searching initiative
Cochrane collaborators around the world hand search

health care journals and conference proceedings to locate
randomized controlled trials of healthcare interventions.
RCTs found are entered into the Cochrane CENTRAL

Register of Controlled Trials.
Currently, over 2300 journals are being searched.

The Master List of hand-searched journals can be

viewed/searched online at
http://apps1.jhsph.edu/cochrane/masterlist.asp

Tracking your Literature Search
Record information on all resources searched (and when),

search terms used, when the search was completed and
how many records/results were found
Track studies retrieved, ordered and received
Consider using a bibliographic management software
program to store and organize search results as well as
track other activities associated with the systematic review
process.
Eg: EndNote, RefWorks, Reference Manager etc..

RevMan
RevMan is the systematic reviews software used by
Cochrane to develop and publish Cochrane reviews
http://www.cc-ims.net/RevMan
References retrieved from literature searching can be
•
imported into RevMan once they have been converted into
plain text records formatted to Vancouver Style
Most bibliographic software programs are capable of
converting references into a RevMan compatible format

Cochrane Collaboration Support
As part of the Cochrane Protocol approval process, a

Trials Coordinator from your Research Group will
advise/assist/review your search strategy
Literature searching support from Cochrane varies with

each Research Group
Researchers may consider approaching a librarian from

their own institution for assistance with their Cochrane
review

Questions? Comments?
Diane Lorenzetti MLS
Centre for Health & Policy Studies
Institute of Health Economics
dllorenz@ucalgary.ca / 210-9319