This talk provided a quick overview of the involvement of two NIH Informationists in an a systematic review, and highlight ways in which other library professionals might incorporate these practices into their systematic review programs.

1. 10/22/2014
1
NIH Library | http://nihlibrary.nih.gov
Analytical Methods for Systematic Review
Support
Doug Joubert - MS, MLIS
Chris Belter - MLS
NIH Library
2014 MAC-MLA Meeting
Outline
Systematic Review
Review
Process
Sci2 Analysis
Sci2 Screening
Conclusions

2. 10/22/2014
2
• Title/abstract reviewing
• First time participating in screening.
• Epi Info™ 7: Free software from the CDC that allows
you to create forms & enter data.
• Science of Science (Sci2) Tool: Open source software
for creating and analyzing publication-based networks.
• Gephi: Open source network visualization software.
Systematic review tools
• Purpose of this systematic review/meta-analysis is:
• to assess the extent to which evidence-based mental
health interventions reduce racial and ethnic disparities in
access to mental health care.
• to identify potential facilitators and barriers to accessing
mental health care among racial and ethnic minorities.
• Access adhered to Penchansky and Thomas (1981)
dimensions of access, including availability, affordability,
accessibility, accommodation, and acceptability.
Systematic review overview

3. 10/22/2014
3
Search strategy and results
• Search strategy was collaboratively developed by
everyone on the systematic review team.
• Set 1: Racial or Ethnic Disparities terms (n=157).
• Set 2: Access Terms (n=75).
• Set 3: Mental Health Services (n=59).
• Set 4: Disorders: depressive disorders, anxiety disorders,
schizophrenia, or bipolar disorder terms (n=68).
• APA PsycNET, Medline, and Scopus databases were
searched:
• APA PsycNET (n=102)
• Medline (n=448)
• Scopus (n=823)
Review process
• First stage of screening involved screening the
title/abstract against the inclusion and exclusion criteria.
• Epi Info™ 7 was used to screen titles/abstracts.
• 4 screeners, worked in pairs.
• Applied inclusion and exclusion factors using screening
form.
• Intercoder reliability (concordance rate) was discussed
during weekly meetings.

4. 10/22/2014
4
Epi Info™ 7 (Slide 1)
Epi Info™ 7 (Slide 2)

5. 10/22/2014
5
Epi Info™ 7 (Slide 3)
Epi Info™ 7 (Slide 4)

6. 10/22/2014
6
Concordance reports
Screening alternatives

7. 10/22/2014
7
Network
Analysis
Identify new
search terms
to find papers
missed by the
initial search
Identify new
search terms
Word co-occurrence
analysis
Identify new
papers
Co-citation
analysis
Background and approaches

8. 10/22/2014
8
What is word co-occurrence?
Paper Title: Economic Stress and Mental Health
Network: Economic Stress
Mental
Health
Word co-occurrence network

9. 10/22/2014
9
What is co-citation?
Article
A
Article
B
Article
C
Co-citation network

10. 10/22/2014
10
• 191 new title/abstracts screened
• 5 new articles classified as “yes”→move to second round
of review
• 18 new articles classified as “?”→move to second round of
review
• 168 new articles classified as “no’”→not included in
second round of review
• 148 (88%) of the no's were rejected because they did not meet the
“condition of interest”
• 13 (8%) of the no's were rejected because they did not meet the
“population of Interest”
• 7 (4%) of the no's were rejected because they did not meet the
“study design”
Sci2 title/abstract screening
Sci2 title/abstract screening
• Of the 23 “yes” or “?” records identified:
• 49 keywords identified
• 41 unique keywords
• 23 new keywords not included in SR search
• 2 might warrant inclusion in an updated search
• Our ratio of (keep/total screened), was compared to the
ratio (keep/total screened) from the actual systematic
review:
• Our ratio: 23/191 = 0.1204 (12.04%)
• Systematic review ratio: 71/1373 =0.0517 (5.17%)

11. 10/22/2014
11
Conclusions
• With proper mentoring, librarians and other information
professionals can participate in title/abstract screening.
• Epi Info™ and SurveyMonkey® are easy to use tools for
developing screening forms.
• Network analysis can identify new keywords and
phrases for search query expansion.
• Co-citation analysis can identify new publications, but is
dependent on the publication set analyzed.
NIH Library | http://nihlibrary.nih.gov
Doug Joubert douglas.joubert@nih.gov
christopher.belter@nih.gov Chris Belter