Presentation for HLG 2018 on peer review of literature searches at Brighton & Sussex NHS Library and Knowledge Service.

1. LITERATURE SEARCHING PEER
REVIEW IN PRACTICE: ENHANCING THE
SKILLS OF SEARCHERS
Rachel Playforth & Igor Brbre
Brighton and Sussex NHS Library & Knowledge Service
HLG Conference 15 June 2018

2. Peer review (noun)
evaluation of scientific, academic,
or professional work by others
working in the same field

3. Background
Brighton and Sussex NHS Library and Knowledge Service
• 3 libraries located at 3 sites
• 29 members of staff, 15 searchers
• Regular peer review of recently completed searches
This project
• Demonstrate how searching peer review works in practice
• Assess the impact
• Areas for improvement

4. Literature review
Pragmatic literature review April 2018. 16 papers selected.
Results in three main categories:
• about the PRESS instrument
• reports on usage of search peer review checklists
• guidelines encouraging peer review of searching
No reports of formal searching peer review practice in
libraries were found.

5. Why we do it
• Continuous service improvement
• Professional development
• Knowledge management and exchange
‘to detect errors, to improve quality, and to reduce not only the risk of missing
relevant studies but also the risk of identifying unnecessarily large numbers of
irrelevant records’ - HTAi Vortal, 2017

6. PRESS
Peer Review of Electronic Search Strategies
2006 - study reveals errors in systematic review search strategies
2007 - CADTH funded study to assess the elements important for peer
review of search strategies
2008 - the first report including PRESS checklist published
2012, 2013 - research shows searching peer review improves quality
2014 - CADTH funds an update of the 2007 study
2016 - revised guideline statement and checklist published

7. PRESS and our adaptation
Our form pre-2015 Our current form PRESS 2015
Search question
Translation into concepts (PICO, followed
up request?)
[searcher supplied question and PICO]
Translation
Sources chosen Sources (strategy adapted?) [searcher supplied; single database only]
Search strategy
Search operators Boolean and proximity operators
Subject headings Subject headings
Natural language Text word searching
Spelling & syntax Spelling, syntax and line numbers
Search filters
Limits and filters
Limits
Search fields utilized [in Text word searching]
Attachments and
cover sheet
Presentation (search notes and summary)
Time spent
CPD needs identified
General comments General comments Overall evaluation

8. How we do it
• Managed through Searching and Teaching Peer Group
• Random pairings, every 4 months
• Review meeting in pairs
• Structured discussion using PRESS-based checklist
• Completed forms anonymised and summarised by chair of S&T Group
• Summary shared and discussed in the S&T group meeting
• Issues and CPD requirements taken forward and actioned

9. What we think of it
• Survey sent to 15 current and 7 former searchers
• Response rate n=11 (50%)
• Anonymous
• Asked about aspects of searching and opinions on peer
review process
• All but 1 thought peer review had made them a better
searcher

10. 0 1 2 3 4 5 6 7 8 9
Sources searched (range and appropriateness)
Proximity operators (ADJ et al.)
Writing search summaries
Using subject headings
Using PICO or other conceptual frameworks
Using search filters
Truncation and wildcards
Using database limits
Other (using KS)
Other (reduction in time spent)
Boolean operators (AND, OR, NOT)
Using database fields
In which areas have you improved?

11. 0 1 2 3 4 5 6 7 8 9
Writing search summaries
Sources searched (range and appropriateness)
Proximity operators (ADJ et al.)
Using search filters
Using database fields
Using database limits
Using PICO or other conceptual frameworks
Using subject headings
Boolean operators (AND, OR, NOT)
Truncation and wildcards
In which areas do you feel you could further improve?

12. Sources
PICO etc
Subject headings
Presentation/summaries
Natural language/truncation etc
Limits
Time taken
Operators
Fields
Filters
Areas discussed in peer review

13. What we think of it
Areas people found most difficult
• The peer review meeting itself
• The summaries discussed in our group meetings
Suggestions
• Do it less frequently
• Review (some) searches before (and after) completion
• Adapt the form for non-database searches
• Make feedback more actionable, develop training

14. What we will do next
• Revise our peer review form
• Discuss peer reviewing (some) searches before
completion

15. Collaboration options
• PRESS Forum
http://pressforum.pbworks.com
• KnowledgeShare

16. What else do we do
• Since 2017 - peer review of teaching
• Journal club

17. Selected bibliography
Sampson M, McGowan J. Errors in search strategies were identified by type and frequency. J Clin
Epidemiol 2006;59(10):1057-63. [accessed 25 April 2018]. Available from:
https://linkinghub.elsevier.com/retrieve/pii/S0895-4356(06)00043-6
Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice
guideline for the peer review of electronic search strategies. J Clin Epidemiol 2009 Sep;62(9):944-52
[accessed 25 April 2018]. Available from: https://www.jclinepi.com/article/S0895-4356(08)00320-
X/fulltext
McGowan, J., Sampson, M., & Lefebvre, C. (2010). An Evidence Based Checklist for the Peer Review
of Electronic Search Strategies (PRESS EBC). Evidence Based Library and Information Practice, 5(1),
149-154 [accessed 25 Apr 2018]. Available from:
https://journals.library.ualberta.ca/eblip/index.php/EBLIP/article/view/7402
Spry C, Mierzwinski-Urban M, Rabb D. Peer review of literature search strategies: does it make a
difference? Presented at Canadian Health Libraries Association (CHLA) Conference; 22-25 May 2013;
Saskatoon, Saskatchewan: Canada [accessed 25 April 2018]. Available from:
https://www.cadth.ca/media/is/CSpry_Poster.pdf
McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of
Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016 Jul;75:40-6 [accessed
25 April 2018]. Available from: https://www.jclinepi.com/article/S0895-4356(16)00058-5/fulltext

18. Contacts
Rachel Playforth
Rachel.playforth@bsuh.nhs.uk
@archelina
Igor Brbre
Igor.brbre@bsuh.nhs.uk
@iroberger
@BrightonSx_LKS