The document provides an overview of peer review, detailing its mechanisms, types, benefits, and challenges in research evaluation and publication. It discusses various forms of peer review, such as single-blind and double-blind, and highlights issues like predatory journals and the effectiveness of the peer review process. Additionally, it emphasizes the importance of transparency and offers resources for potential peer reviewers.

1. Introduction to peer review
Peer review workshop, Wageningen Graduate Schools
Wouter Gerritsma

2. Contents
 What is peer review?
 Peer review problems
 Peer review solutions
 Peer review benefits
 Peer review resources

3. Nature editorial Februari 19th, 2015
http://dx.doi.org/10.1038/518274b

4. What is peer review?
Peer review is both a set of mechanisms and a
principle at the heart of the system for evaluating and
assuring the quality of research before and after it is
funded or published. It involves subjecting research
proposals and draft presentations, papers and other
publications to critical evaluation by independent
experts (peers). The reviewers are usually appointed
by the funding body or the editors
of a journal or other formal channel for
communication to which the work has been
submitted.
Source: RIN (2011). Peer review: a guide for researchers.
www.rin.ac.uk/peer-review-guide.

5. When is peer review employed?
 the evaluation of applications for funding, to determine which
applications are successful
 the review of reports submitted by researchers once their
funding award has come to an end, to assess whether a
project has been completed satisfactorily
 the evaluation of draft conference presentations, journal
articles and monographs, before they are published, to assess
whether they meet quality standards
 the evaluation of publications once they have been published,
through reviews and review articles
 the evaluation of the quality of work produced by individuals,
teams, departments and institutions to help determine
appointments, promotions and levels of funding.

6. Peer review of publications I

7. Peer review of publications II

8. Which editor?
 Roles of 113 WageningenUR researchers with 71 Elsevier
journals
● Editor in chief
● Editor
● Reviews editor
● Associate editor
● Editorial board
● Section editor
● Topic editor
● Editorial advisory board
● Book review editor

9. Einstein to the editor of Physical Review
Kennefick, D. (2005). Einstein Versus the Physical Review. Physicstoday, 58(9): 43
http://physicstoday.org/journals/doc/PHTOAD-ft/vol_58/iss_9/43_1.shtml

10. Forms of peer review
 Editorial peer review: The editors themselves judge weather a submission
is accepted for publication
 Single-blind review: the identities of those who have submitted the
proposal or draft publication are revealed to the reviewers, but not vice versa
 Double-blind review: the identities of the reviewers and those whose
submission is being reviewed are hidden from each other.
 Open peer review: this term is used to cover at least three different kinds of
arrangement with increasing levels of transparency (eg. HESS):
● the identities of reviewers and submitters are revealed to each other
● the signed reviews themselves are passed in full to the applicants, and
● authors’ draft publications are made available on websites and reviews
and comments are invited from anyone who wishes to do so.

11. Effectiveness of different types of peer review
Types of peer review % agree
(n=4037)
Peer review could be replaced by usage
statistics
15%
Supplementing peer review with post
publication review
47%
Open and published peer review 25%
Open peer review 20%
Double blind peer review 76%
Single blind peer review 45%
Mulligan, A., Hall, L., & Raphael, E. (2013). Peer review in a changing
world: An international study measuring the attitudes of researchers.
JASIST, 64(1), 132-161. http://dx.doi.org/10.1002/asi.22798

12. Transparency of the review proces
 Do the journals encourage suggestions for reviewers:
http://www.plosone.org/article/info:doi/10.1371/journal.pone.0013345
 Clear dates of submission revised accepted published
http://link.springer.com/article/10.1007/s10750-012-1234-6/fulltext.html
 Indication of the handeling editor http://link.springer.com/article/10.1007/s10750-
012-1234-6
 Thanking the peer reviewers once a year
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0121093
 Open Peer Review: http://bmjopen.bmj.com/content/3/9/e003272

13. Predatory journals
 They don't look all like Antarctica Journal of Mathematics
 Fake editorial boards (are they credible scientists?)
 Very quick/consistent period from submission to
acceptance (no date for revision!)
 No language editing/poor English
 Quality of the articles
 Beall's list of predatory publishers
http://scholarlyoa.com/publishers

14. Recent developments
Post publication peer review
 PubPeer https://pubpeer.com/publications/20129177
 F1000 Prime http://f1000.com/prime/recommendations
PeerJ, marrying publishing and peer review
Cascading peer review

15. The publishing industry: publishers view

16. There is still no sign of decline
Larsen, P. & M. von Ins (2010). The rate of growth in scientific publication and the decline in
coverage provided by Science Citation Index. Scientometrics, 84(3): 575-603
http://dx.doi.org/10.1007/s11192-010-0202-z

17.  In 2006 about 1,350,000 articles were published in
peer-reviewed journals (WoS)
 2,000,000 in 2013 (Scopus)
 Drivers
● More scientists, more
publications
● Pressure to publish (in
journals)
● In search for the "least
publishable unit"

18. Other criticism
 Not always effective at detecting falsification, fabrication
and plagiarism (FFP)
 Brings delay in the research an innovation cycle
 Selection of reviewers brings bias
 Judgement subjective and inconsistent
 Tends toward conservatism and stifles innovation
 Disadvantageous to interdisciplinary research
 Imposes increasing and unsupportable burdens on
reviewers

19. Retractions are related to journal prestige
Fang, F.C. & A. Casadevall (2011). Retracted science and the retraction index.
Infection and Immunity, http://dx.doi.org/10.1128/iai.05661-11

20. Retractions on the rise
Van Noorden, R. (2011). Science publishing: The trouble with retractions.
Nature, 478: 26-28 http://dx.doi.org/10.1038/478026a

22. Peer review, a plan of action
Nicholas, K. A. & W. S. Gordon (2011). A quick guide to writing a solid peer review.
EOS, Transactions American Geophysical Union, 92(28): 233-234
http://dx.doi.org/10.1029/2011eo280001

23. Considering a reviewing request
 Do you have the expertise?
 Is there a conflict of interest?
 Do you have time?

24. After the review process
 You should be informed on the editor's decision
● If the editor makes a decision on the manuscript
counter to the direction you recommended in your
review, you may request an explanation.
 You should not reveal to the author or authors after
review that you were a reviewer
 Do not make public the contents of the manuscript nor
use any information in the manuscript until it is
published.

25. Why should you take part?
 New peer reviewers badly needed
 You can benefit from peer review
● Critical reading
● Expressing your opinion
● Improve you own writing
● Expand your professional network

26. Some resources
 BMJ peer reviewers: resources
http://www.bmj.com/about-bmj/resources-reviewers
 Elsevier's reviewers home
http://www.elsevier.com/wps/find/reviewershome.reviewers
 Springer peer review academy
http://www.springer.com/authors/journal+authors/peer-review-academy
 Nature peer review debate
http://www.nature.com/nature/peerreview/debate/index.html
 British Ecological Society http://www.britishecologicalsociety.org/wp-
content/uploads/Publ_Peer-Review-Booklet.pdf

27. Thank you!
On the Web:
@wowter
wowter.net
This presentation
http://www.slideshare.net/Wowter/introduction-to-peer-review
http://tinyurl.com/7r67fmm