Nominated for the Thomas C Chalmers Award

1. 1. Kastner M, Wilczynski NL, Walker-Dilks C, McKibbon KA, Haynes B. Age-specific search strategies for Medline
J Med Internet Res. 2006 Oct 25;8(4):e25
For more information, please contact Edith Leclercq: e.leclercq@amc.uva.nl
Validation of a PubMed search filter for identifying studies including children
Edith Leclercq1, Mariska MG Leeflang2, Elvira C van Dalen¹, Leontien CM Kremer¹
¹Cochrane Childhood Cancer Group, Department of Paediatric Oncology, Emma Children's Hospital (EKZ) / Academic Medical Center, Amsterdam (AMC), the Netherlands; ²Dutch Cochrane Center,
Academic Medical Centre, Amsterdam, the Netherlands
For physicians and other medical professionals, it is important to find relevant information on their field of
interest. Various search filters, especially for identifying clinical trials and systematic reviews, have been
developed. The aim of this study was to develop a PubMed search filter for identifying studies including
children for the Cochrane Childhood Cancer Group (CCG), and to validate this search filter.
The CCG search filter has been developed by identifying keywords and MeSH headings from relevant
publications and by discussions with experts in the field. We validated the search filter by calculating the
sensitivity and precision, and compared these results with six other known search filters for children (PubMed
Limit All Child, three by Kastner et al. (ref.1), Child Health Field (CHF), and BestBets).
A new Gold Standard was developed by collecting all randomized controlled trials (RCTs) and controlled
clinical trials (CCTs) reported in Cochrane intervention reviews that included children (1951–2008). This Gold
Standard was stored in PubMed. Sensitivity was calculated by the number of retrieved relevant RCTs and
CCTs divided by the Gold Standard set of relevant studies.
The precision of the 7 filters was calculated as Number Needed to Read (NNR) in a set of records focussed
on children obtained from PubMed in September 2008. All records were assessed by two authors individually
to determine whether a study included children or not. In case of disagreement a third author was consulted.
The five child search filters from CCG, CHF, BestBets, Kastner1 (filter for 'best sensitivity') and PubMed
showed a high sensitivity to detect studies including children. Two of the three child search filters from
Kastner et al. (Kastner2, Kastner3) performed less well in PubMed (Table 1).
The NNR was calculated for all child filters and varied from 1.06 (Kastner3) to 1.52 (CCG) (Table 2). Two
Kastner filters (Kastner2, Kastner3), seem to be very efficient (NNR 1.07 and 1.06 respectively). However,
they do miss the highest number of relevant studies (173 and 230 respectively of the 416 relevant records)
(Table 2).
False positive records with the CCG search filter were obtained amongst others with the search terms boy*
and school*.
Abbreviations: GS= Gold Standard; CCG=Childhood Cancer Group; CHF=Child Health Field; NNR=Number Needed to
Read; Kastner1= search filter for best sensitivity; Kastner2=search filter for best specificity; Kastner3=search filter for best
optimization of sensitivity and specificity; n.a.=not applicable
Table 1. Sensitivity of child search filters in PubMed
Table 2. NNR of child search filters in PubMed
Background
Conclusions
Methods
Results
All search filters for identifying studies including children performed rather similar, except for two search filters
developed by Kastner et al (Kastner2, Kastner3), which performed less well. Based on the results of this study
recommendations can be made for the improvement of the CCG child filter.
Retrieved references
Paediatric search
filter
Total Relevant Missed NNR
CCG 628 413 3 1.52
CHF 593 414 2 1.43
BestBets 611 413 3 1.48
Kastner1 526 394 22 1.34
Kastner2 261 243 173 1.07
Kastner3 198 186 230 1.06
PubMed limit All
Child: 0-18 years
514 392 24 1.31
Search filter Number of hits Sensitivity (%) 95% CI
GS 1357 n.a. n.a.
CCG + GS 1332 98.16 0.97-0.99
CHF + GS 1331 98.08 0.97-0.99
Best Bets + GS 1333 98.23 0.97-0.99
Kastner1 + GS 1324 97.57 0.97-0.98
Kastner2 + GS 629 46.35 0.44-0.49
Kastner3 + GS 735 54.16 0.51-0.57
PubMed limit All Child: 0-
18 years + GS
1320 97.27 0.96-0.98