![Systematic searching?...well, we had a bit of a look
C urrently peer reviewed journals are peppered
with so-called systematic reviews. However,
on closer inspection it becomes obvious that the
search strategies used to locate evidence for many
of them are anything but systematic. Database search-
ing is the initial investigative stage of any systematic
review following the development of the protocol and
is analogous to laying the foundations of a building;
however magnificent the architectural conception, if
thefoundationsarenotsound,the entirestructurewill
be at risk, and although the foundations may be the
most unexciting part of the project, they are crucial to
the integrity of the building. My favourite article on
how not to search is, ‘‘A surrealistic mega-analysis of
redisorganization theories’’,1
a wonderful send-up
of evidence based medicine by David Sackett and
others. Unfortunately, it appears that these or similar
methods are often applied in a more serious context.
Let’s look at some of the most frequently published
faux pas in so-called systematic review searches.
The first is commonly found in the list of
databases searched. ‘‘We searched PubMed,
MEDLINE. . .’’ PubMed includes MEDLINE,2
so
listing both indicates that neither the authors nor
the peer reviewers were aware of this. Not a good
start to establishing credibility. MEDLINE is also
available on other platforms such as Ovid, however
the content is identical to that used in PubMed; it is
simply leased from the National Library of Medicine
and made available on the new provider’s platform.
A second common fault is the inclusion of Science-
Direct which often appears in the list of databases
searched, but is essentially a searchable collection of
journals and books from specific publishers. The
content is therefore limited, and the search function
lacks sophistication. Databases, on the other hand,
are not limited by publisher, and are generally
designed to allow highly refined searching. Google
Scholar is another frequently cited source, and while
it certainly has its uses, it does not allow complex
searching, nor can the results obtained from a Google
Scholar search be reliably replicated
Next comes the description of the search under-
taken. ‘‘We searched [list of databases] . . .using the
following keywords. . .’’ This indicates that neither
the searchers nor the peer reviewers understood
that different databases have different indexing
languages, and may use quite different terms for
the same concept. For example PubMed’s indexing
term for ‘‘heart attack’’ is ‘‘myocardial infarction’’,
whereas Embase uses the indexing term ‘‘heart
infarction’’. A search suited to one database cannot
simply be transferred to another. This example also
demonstrates ignorance of the range of potential
fields that can be searched in many databases – titles
of articles, abstracts of articles, authors’ keywords,
and in some cases the full text of articles. Failure to
specify a search field often means that the database’s
algorithm generates bizarre results by interpreting
the search in a completely unexpected manner.3
‘‘We
searched PubMed using the following MeSH (index-
ing terms). . .’’ Although this search method sounds
more convincing, unless correct syntax is used, the
search may not work as anticipated.4
Secondly, it
does not allow for the possibility of indexer error, or
variation in the way indexers may interpret the
content of an article. Restricting searching to MeSH
alone fails to allow for searching of the hundreds of
thousands of new, unindexed articles supplied to
PubMed by the various publishers.
The same principle applies to other databases
where new material will be unindexed, and there-
fore accessible only by searching in titles, abstracts
or authors’ keywords. In Embase, new material is
computer indexed, using thesaurus (Emtree) terms,
and later checked for accuracy by indexing staff.
Recently, I worked with a higher degree student
who was researching oropharyngeal dysphagia. I
was dismayed to see she had included in her search
the Emtree term ‘‘swallow (bird)’’. She explained
that she also thought this rather odd, but had seen
the indexing term attached to relevant articles, and
therefore felt obliged to include it. On further
investigation we discovered that this was in fact
a computer error, but one that had remained
undetected by indexers for a number of years.
The Emtree term has now been changed to ‘‘swal-
low’’ with ‘‘(bird)’’ no longer appearing, but sadly,
despite my correspondence with Elsevier technical
staff in Frankfurt, the computer indexing error
persists.
Database searching in systematic reviews is a
demanding task, and one which requires rigorous
testing of each individual component of the search,
JBI Database of Systematic Reviews and Implementation Reports ß 2018 THE JOANNA BRIGGS INSTITUTE 258
EDITORIAL
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.](https://image.slidesharecdn.com/systematicsearching-190905222238/85/Systematic-searching-well-we-had-a-bit-of-a-look-1-320.jpg)
![as well as careful attention to its structural and
functional elements. It is an exercise involving both
logic and linguistics, and while modern technology
has been a boon, researchers would do well to
remember that the artificial intelligence which gen-
erates database algorithms is rather like artificial
flavors. It’s a valuable asset, but no substitute for
the real thing. Effective critical analysis and decision-
making in this context remain very much the prov-
ince of the human brain and its intelligence. So it’s
really up to you.
Maureen Bell
Research Librarian, University of Adelaide,
Adelaide South Australia
DOI: 10.11124/JBISRIR-2017-003674
References
1. Oxman AD1, Sackett DL, Chalmers I, Prescott TE. A surrealistic
mega-analysis of redisorganization theories. J R Soc Med. 2005;
98(12):563–568. [Internet] [cited 2017 Oct 26]. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1299350/.
2. MEDLINE, PubMed, and PMC (PubMed Central): How are they
different? Fact Sheet, National Library of Medicine [Internet].
2002 July 12 [Updated 2017 Sept 8; cited 2017 Oct 26].
Available from: https://www.nlm.nih.gov/pubs/factsheets/
dif_med_pub.html.
3. Bell M. PubMed - Why Use Field Names? [video]. 2017 [cited
2017 Oct 26]. Available from: https://www.youtube.com/
watch?v=ZTnbouR1gPc.
4. Bell M. Taming MeSH or Who’s in charge here??!! [video].
2017 [cited 2017 Oct 26]. Available from: https://www.
youtube.com/watch?v=CAUUSr5ALf4.
EDITORIAL M. Bell
JBI Database of Systematic Reviews and Implementation Reports ß 2018 THE JOANNA BRIGGS INSTITUTE 259
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Systematic searching...well we had a bit of a look
Similar to Systematic searching...well we had a bit of a look (20)
Recently uploaded
Recently uploaded (20)
Systematic searching...well we had a bit of a look
- 1. Systematic searching?...well, we had a bit of a look C urrently peer reviewed journals are peppered with so-called systematic reviews. However, on closer inspection it becomes obvious that the search strategies used to locate evidence for many of them are anything but systematic. Database search- ing is the initial investigative stage of any systematic review following the development of the protocol and is analogous to laying the foundations of a building; however magnificent the architectural conception, if thefoundationsarenotsound,the entirestructurewill be at risk, and although the foundations may be the most unexciting part of the project, they are crucial to the integrity of the building. My favourite article on how not to search is, ‘‘A surrealistic mega-analysis of redisorganization theories’’,1 a wonderful send-up of evidence based medicine by David Sackett and others. Unfortunately, it appears that these or similar methods are often applied in a more serious context. Let’s look at some of the most frequently published faux pas in so-called systematic review searches. The first is commonly found in the list of databases searched. ‘‘We searched PubMed, MEDLINE. . .’’ PubMed includes MEDLINE,2 so listing both indicates that neither the authors nor the peer reviewers were aware of this. Not a good start to establishing credibility. MEDLINE is also available on other platforms such as Ovid, however the content is identical to that used in PubMed; it is simply leased from the National Library of Medicine and made available on the new provider’s platform. A second common fault is the inclusion of Science- Direct which often appears in the list of databases searched, but is essentially a searchable collection of journals and books from specific publishers. The content is therefore limited, and the search function lacks sophistication. Databases, on the other hand, are not limited by publisher, and are generally designed to allow highly refined searching. Google Scholar is another frequently cited source, and while it certainly has its uses, it does not allow complex searching, nor can the results obtained from a Google Scholar search be reliably replicated Next comes the description of the search under- taken. ‘‘We searched [list of databases] . . .using the following keywords. . .’’ This indicates that neither the searchers nor the peer reviewers understood that different databases have different indexing languages, and may use quite different terms for the same concept. For example PubMed’s indexing term for ‘‘heart attack’’ is ‘‘myocardial infarction’’, whereas Embase uses the indexing term ‘‘heart infarction’’. A search suited to one database cannot simply be transferred to another. This example also demonstrates ignorance of the range of potential fields that can be searched in many databases – titles of articles, abstracts of articles, authors’ keywords, and in some cases the full text of articles. Failure to specify a search field often means that the database’s algorithm generates bizarre results by interpreting the search in a completely unexpected manner.3 ‘‘We searched PubMed using the following MeSH (index- ing terms). . .’’ Although this search method sounds more convincing, unless correct syntax is used, the search may not work as anticipated.4 Secondly, it does not allow for the possibility of indexer error, or variation in the way indexers may interpret the content of an article. Restricting searching to MeSH alone fails to allow for searching of the hundreds of thousands of new, unindexed articles supplied to PubMed by the various publishers. The same principle applies to other databases where new material will be unindexed, and there- fore accessible only by searching in titles, abstracts or authors’ keywords. In Embase, new material is computer indexed, using thesaurus (Emtree) terms, and later checked for accuracy by indexing staff. Recently, I worked with a higher degree student who was researching oropharyngeal dysphagia. I was dismayed to see she had included in her search the Emtree term ‘‘swallow (bird)’’. She explained that she also thought this rather odd, but had seen the indexing term attached to relevant articles, and therefore felt obliged to include it. On further investigation we discovered that this was in fact a computer error, but one that had remained undetected by indexers for a number of years. The Emtree term has now been changed to ‘‘swal- low’’ with ‘‘(bird)’’ no longer appearing, but sadly, despite my correspondence with Elsevier technical staff in Frankfurt, the computer indexing error persists. Database searching in systematic reviews is a demanding task, and one which requires rigorous testing of each individual component of the search, JBI Database of Systematic Reviews and Implementation Reports ß 2018 THE JOANNA BRIGGS INSTITUTE 258 EDITORIAL ©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
- 2. as well as careful attention to its structural and functional elements. It is an exercise involving both logic and linguistics, and while modern technology has been a boon, researchers would do well to remember that the artificial intelligence which gen- erates database algorithms is rather like artificial flavors. It’s a valuable asset, but no substitute for the real thing. Effective critical analysis and decision- making in this context remain very much the prov- ince of the human brain and its intelligence. So it’s really up to you. Maureen Bell Research Librarian, University of Adelaide, Adelaide South Australia DOI: 10.11124/JBISRIR-2017-003674 References 1. Oxman AD1, Sackett DL, Chalmers I, Prescott TE. A surrealistic mega-analysis of redisorganization theories. J R Soc Med. 2005; 98(12):563–568. [Internet] [cited 2017 Oct 26]. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1299350/. 2. MEDLINE, PubMed, and PMC (PubMed Central): How are they different? Fact Sheet, National Library of Medicine [Internet]. 2002 July 12 [Updated 2017 Sept 8; cited 2017 Oct 26]. Available from: https://www.nlm.nih.gov/pubs/factsheets/ dif_med_pub.html. 3. Bell M. PubMed - Why Use Field Names? [video]. 2017 [cited 2017 Oct 26]. Available from: https://www.youtube.com/ watch?v=ZTnbouR1gPc. 4. Bell M. Taming MeSH or Who’s in charge here??!! [video]. 2017 [cited 2017 Oct 26]. Available from: https://www. youtube.com/watch?v=CAUUSr5ALf4. EDITORIAL M. Bell JBI Database of Systematic Reviews and Implementation Reports ß 2018 THE JOANNA BRIGGS INSTITUTE 259 ©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.