Introduction to Peer review, updated 2015-03-05


Published on

Introduction to the peer review workshop for the PhD students of the Wageningen Graduate Schools. The goal is to explain peer review, entice PhD students to take part in the peer review process and give some tips on how to start with peer review.

Introduction to Peer review, updated 2015-03-05

  1. 1. Introduction to peer review Peer review workshop, Wageningen Graduate Schools Wouter Gerritsma
  2. 2. Contents  What is peer review?  Peer review problems  Peer review solutions  Peer review benefits  Peer review resources
  3. 3. Nature editorial Februari 19th, 2015
  4. 4. What is peer review? Peer review is both a set of mechanisms and a principle at the heart of the system for evaluating and assuring the quality of research before and after it is funded or published. It involves subjecting research proposals and draft presentations, papers and other publications to critical evaluation by independent experts (peers). The reviewers are usually appointed by the funding body or the editors of a journal or other formal channel for communication to which the work has been submitted. Source: RIN (2011). Peer review: a guide for researchers.
  5. 5. When is peer review employed?  the evaluation of applications for funding, to determine which applications are successful  the review of reports submitted by researchers once their funding award has come to an end, to assess whether a project has been completed satisfactorily  the evaluation of draft conference presentations, journal articles and monographs, before they are published, to assess whether they meet quality standards  the evaluation of publications once they have been published, through reviews and review articles  the evaluation of the quality of work produced by individuals, teams, departments and institutions to help determine appointments, promotions and levels of funding.
  6. 6. Peer review of publications I
  7. 7. Peer review of publications II
  8. 8. Which editor?  Roles of 113 WageningenUR researchers with 71 Elsevier journals ● Editor in chief ● Editor ● Reviews editor ● Associate editor ● Editorial board ● Section editor ● Topic editor ● Editorial advisory board ● Book review editor
  9. 9. Einstein to the editor of Physical Review Kennefick, D. (2005). Einstein Versus the Physical Review. Physicstoday, 58(9): 43
  10. 10. Forms of peer review  Editorial peer review: The editors themselves judge weather a submission is accepted for publication  Single-blind review: the identities of those who have submitted the proposal or draft publication are revealed to the reviewers, but not vice versa  Double-blind review: the identities of the reviewers and those whose submission is being reviewed are hidden from each other.  Open peer review: this term is used to cover at least three different kinds of arrangement with increasing levels of transparency (eg. HESS): ● the identities of reviewers and submitters are revealed to each other ● the signed reviews themselves are passed in full to the applicants, and ● authors’ draft publications are made available on websites and reviews and comments are invited from anyone who wishes to do so.
  11. 11. Effectiveness of different types of peer review Types of peer review % agree (n=4037) Peer review could be replaced by usage statistics 15% Supplementing peer review with post publication review 47% Open and published peer review 25% Open peer review 20% Double blind peer review 76% Single blind peer review 45% Mulligan, A., Hall, L., & Raphael, E. (2013). Peer review in a changing world: An international study measuring the attitudes of researchers. JASIST, 64(1), 132-161.
  12. 12. Transparency of the review proces  Do the journals encourage suggestions for reviewers:  Clear dates of submission revised accepted published  Indication of the handeling editor 012-1234-6  Thanking the peer reviewers once a year  Open Peer Review:
  13. 13. Predatory journals  They don't look all like Antarctica Journal of Mathematics  Fake editorial boards (are they credible scientists?)  Very quick/consistent period from submission to acceptance (no date for revision!)  No language editing/poor English  Quality of the articles  Beall's list of predatory publishers
  14. 14. Recent developments Post publication peer review  PubPeer  F1000 Prime PeerJ, marrying publishing and peer review Cascading peer review
  15. 15. The publishing industry: publishers view
  16. 16. There is still no sign of decline Larsen, P. & M. von Ins (2010). The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index. Scientometrics, 84(3): 575-603
  17. 17.  In 2006 about 1,350,000 articles were published in peer-reviewed journals (WoS)  2,000,000 in 2013 (Scopus)  Drivers ● More scientists, more publications ● Pressure to publish (in journals) ● In search for the "least publishable unit"
  18. 18. Other criticism  Not always effective at detecting falsification, fabrication and plagiarism (FFP)  Brings delay in the research an innovation cycle  Selection of reviewers brings bias  Judgement subjective and inconsistent  Tends toward conservatism and stifles innovation  Disadvantageous to interdisciplinary research  Imposes increasing and unsupportable burdens on reviewers
  19. 19. Retractions are related to journal prestige Fang, F.C. & A. Casadevall (2011). Retracted science and the retraction index. Infection and Immunity,
  20. 20. Retractions on the rise Van Noorden, R. (2011). Science publishing: The trouble with retractions. Nature, 478: 26-28
  21. 21. Peer review, a plan of action Nicholas, K. A. & W. S. Gordon (2011). A quick guide to writing a solid peer review. EOS, Transactions American Geophysical Union, 92(28): 233-234
  22. 22. Considering a reviewing request  Do you have the expertise?  Is there a conflict of interest?  Do you have time?
  23. 23. After the review process  You should be informed on the editor's decision ● If the editor makes a decision on the manuscript counter to the direction you recommended in your review, you may request an explanation.  You should not reveal to the author or authors after review that you were a reviewer  Do not make public the contents of the manuscript nor use any information in the manuscript until it is published.
  24. 24. Why should you take part?  New peer reviewers badly needed  You can benefit from peer review ● Critical reading ● Expressing your opinion ● Improve you own writing ● Expand your professional network
  25. 25. Some resources  BMJ peer reviewers: resources  Elsevier's reviewers home  Springer peer review academy  Nature peer review debate  British Ecological Society content/uploads/Publ_Peer-Review-Booklet.pdf
  26. 26. Thank you! On the Web: @wowter This presentation