Who reviews for predatory journals presentation

1. -
Who reviews for predatory journals?
A study on reviewer characteristics
Anna Severin1,2, Michaela Strinzel1, Matthias Egger1,2, Marc Domingo3, Tiago Barros4
1 Swiss National Science Foundation (SNSF), Bern, Switzerland
2 University of Bern, Bern, Switzerland
3 Clarivate, London, UK
4 Faculty Opinions LTD, London, UK

2. SNSF 19.11.2020 – 2 Research creates knowledge.
-
Investigate profiles of scholars who have submitted reviews for predatory journals
and legitimate journals to Publons, in terms of reviewing and publishing behaviour
and geographical location.
Rationale
• Predatory journals prioritize self-interest at the expense of scholarship and are characterized by
false information, deviation from best editorial and publication practices, a lack of transparency,
and/or the use of aggressive and indiscriminate solicitation practices (Grudniewicz et al., 2019)
• It is widely assumed that
• Most predatory journals do not conduct peer review.
• If predatory journals do conduct peer review, it lacks scientific integrity (anecdotal
evidence of publishers ignoring reviewer publication recommendations)
Background

3. SNSF 19.11.2020 – 3 Research creates knowledge.
-
Materials and Methods

4. SNSF 19.11.2020 – 4 Research creates knowledge.
-
Clarivate’s Publons product
• A platform for researchers to track and get credit for their peer review.
• In December 2019, Publons contained data on more than 5 million reviews,
spanning more than 500000 reviewers and approximately 40000 journals.
Cabells Scholarly Analytics’ journal lists
• Scholarly services company that maintains a list of legitimate journals and a list of
predatory journals.
• The list of predatory journals and the list of legitimate journals were downloaded in
December 2018 and contained 10671 and 11057 unique journal titles, respectively.
Materials

5. SNSF 19.11.2020 – 5 Research creates knowledge.
-
Methods
Figure 2: Procedure of identifying reviewers for predatory journals and legitimate journals (Severin et al. 2020)

6. SNSF 19.11.2020 – 6 Research creates knowledge.
-
For reviewers of matched reviews,
(1) Meta-data was retrieved
• Main institutional affiliation
• Dates of first publication and last publication
• Number of publications
• Number of reviews in Publons database
(2) Proxies for academic age and academic productivity were calculated
• Academic age: Date of last publication – date of last publication
• Publishing productivity (number of publications / years of publishing)
• Reviewing productivity (number of reviews / years of publishing)
Methods

7. SNSF 19.11.2020 – 7 Research creates knowledge.
-
Results

8. SNSF 19.11.2020 – 8 Research creates knowledge.
-
General patterns
Results
Figure 2: Shares of reviews, n =183,743 unique verified Publons reviews

9. SNSF 19.11.2020 – 9 Research creates knowledge.
-
Reviewer subgroups and their characteristics
Results
Subgroup No. of reviewers
(share of all
reviewers)
Mean
academic age
(SD)
Mean no. of
reviews
(SD)
Mean no. of
publications
(SD)
Reviewing
productivity
(SD)
Publishing
productivity
(SD)
No predatory reviews
(0%)
6611 (90.0%) 15.27 (10.28) 46 (77.85) 57.28 (73.71) 3.65 (6.81) 3.25 (3.68)
Few predatory reviews
(1-25%)
555 (7.6%) 17.37 (11.39) 132.3 (153.63) 85.39 (119.08) 9.9 (19.27) 4.43 (3.87)
Some predatory reviews
(26-75%) 138 (1.9%) 12.47 (10.42) 107.8 (173.96) 49.58 (75.26) 10.60 (19.64) 3.42 (3.33)
Many predatory reviews
(76-99%) 19 (0.26%) 8.16 (6.53) 236.6 (401.20) 43.53 (75.74) 36.71 (60.29) 5.29 (12.9)
Predatory reviews only
(100%)
26 (0.35%) 9.38 (6.27) 26.35 (40.82) 18 (14.49) 3.34 (5.64) 1.91 (1.21)
Table 1: Reviewer subgroups and their characteristics

10. SNSF 19.11.2020 – 10 Research creates knowledge.
-
Geographical distribution of predatory reviews
Results
Figure 1: Shares of predatory reviews by geographic region Figure 2: Shares of predatory reviews by World Bank income region

11. SNSF 19.11.2020 – 11 Research creates knowledge.
-
Conclusion

12. SNSF 19.11.2020 – 12 Research creates knowledge.
-
Reviewing patterns
• Many predatory journals do exercise some form of peer review.
• Most reviewers have never submitted a review for a predatory journal.
• A small number of reviewers have submitted reviews for predatory journals only, these
reviewers are of young academic age, inexperienced and located in developing countries
Limitations
• Potential false positives and false negatives
• Small groups of reviewers -> Limited generalizability
• Self-selection biases in claiming reviews on Publons
• Journal status ≠ peer review quality
Conclusion

13. SNSF 19.11.2020 – 13 Research creates knowledge.
-
More information
Richard van Noorden (2020): Hundreds of scientists have peer-reviewed for predatory
journals, Nature News Article, https://www.nature.com/articles/d41586-020-00709-x
Anna Severin, Michaela Strinzel, Matthias Egger, Marc Domingo, Tiago Barros (2020):
Who reviews for predatory journals? A study on reviewer characteristics,
bioRxiv; doi: https://doi.org/10.1101/2020.03.09.983155
Anna Severin
Swiss National Science Foundation
anna.severin@snf.ch
Twitter: annasvrn