Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

OSFair2017 Workshop | PEERE - Research on peer review

132 views

Published on

Jadranka Stojanovski presents PEERE and shares the editors' opinions on Open Peer Review (Croatian OA journals) | OSFair2017 Workshop

Workshop title: Peer review at the crossroads

Workshop overview:
The workshop builds on the results of the OpenUp landscape scan and the OpenAIRE report on open peer review. The workshop has multiple purposes including (1) assessing existing and evolving methods and functions of alternative peer review mechanisms, (2) breaking down peer review into the basic processes to identify the benefits and challenges, and (3) identifying questions and issues that need further investigation.

Group discussions will also touch upon issues such as the sustainability, long-term availability of alternative review tools, and their uptake by researchers, and the incorporation of these methods into institutional, national, funders’ and publishers’ policies.

OpenUP and OpenAIRE are dedicated to engage with different (disciplinary, inter-disciplinary) research communities from the social sciences, life sciences, energy, arts and humanities to identify the requirements from the emerging trends as posed by Open Science and e-infrastructural interconnected environments. Both projects aim at developing a sustainable framework that is relevant for and responsive to the Open Science needs.

DAY 3 - PARALLEL SESSION 6

Published in: Science
  • Be the first to comment

  • Be the first to like this

OSFair2017 Workshop | PEERE - Research on peer review

  1. 1. PEERE - RESEARCH ON PEER REVIEW EDITORS’ OPINIONS ON OPEN PEER REVIEW (CROATIAN OA JOURNALS) Jadranka Stojanovski University of Zadar / Ruđer Bošković Institute Open Science Fair 2017, 6-8 September, Athens, Greece
  2. 2. • Peer reviewers don’t agree much (1899 articles) • Reviewers don’t reliably select highly cited articles (Bornmann & Daniel Angew Chem 2008;47:7173-8) • Peer review is not effective at detecting errors (Godlee et al. The impact of blinding and masking on the quality of peer review. JAMA 1998;280:237-40) • Rigorous peer-review has not prevented the publication of fraudulent science in major journals • slow, expensive, biased, unreliable, open to abuse (Wager & Jefferson, Learned Pub 2001;14:257-63) Open Science Fair 2017, 6-8 September, Athens, Greece Liz Wager, PUBMET, Zadar, 2014
  3. 3. New Frontiers of Peer Review (PEERE) • Trans-Domain COST Action TD1306 • 12/5/2014-11/5/2018 • Chair: Prof Flaminio SQUAZZONI • http://www.peere.org/ • „to improve efficiency, transparency and accountability of peer review through a trans-disciplinary, cross-sectorial collaboration” Open Science Fair 2017, 6-8 September, Athens, Greece
  4. 4. 0 50 100 150 200 250 300 350 1960 1970 1980 1990 2000 2010 2020 # papers on peer review (Scopus) Open Science Fair 2017, 6-8 September, Athens, Greece
  5. 5. Objectives • to analyse peer review in different scientific areas by integrating quantitative and qualitative research; • to evaluate implications of different models of peer review and to explore new incentive structures, rules and measures; • to involve science stakeholders in data sharing and testing initiatives; • to define collaboratively a joint research agenda that points to an evidence-based peer review reform. Open Science Fair 2017, 6-8 September, Athens, Greece
  6. 6. BENEFITS OF OPEN PEER REVIEW • author can see who reviewed their work • reviewer comments put paper in the context which is useful additional information for readers • reduces bias among reviewers • more constructive reviews • published report can serve as peer review examples for young researchers • shows the reviewer’s informed opinion on the work • demonstrates experience of the reviewer • can take credit for the work involved in conducting the review Open Science Fair 2017, 6-8 September, Athens, Greece Squazzone, 2016
  7. 7. OPEN PEER REVIEW RESEARCH • The potential drawbacks of open peer review. A simulation model of scientist behaviour (Squazzoni & Bianchi) • Getting it transparent or keeping it obscure? Potential implications of open peer review on scientist competition and collaboration (Federico Bianchi and Flaminio Squazzoni) • http://www.peere.org/dissemination/presentations/ Open Science Fair 2017, 6-8 September, Athens, Greece
  8. 8. Collaboration with big publishers • PEERE Data Sharing Protocol: regulate data sharing on peer review in all PEERE activities • the document has been completed on 23 December 2016 with Elsevier, Springer and Wiley representatives, and has been signed by all partners on 1 March 2017 Open Science Fair 2017, 6-8 September, Athens, Greece
  9. 9. hrcak.srce.hr Open Science Fair 2017, 6-8 September, Athens, Greece
  10. 10. THE SURVEY ON PEER REVIEW Open Science Fair 2017, 6-8 September, Athens, Greece 441 OA journals 218 with peer review 51 STM 78 SSH 39 questions
  11. 11. QUESTIONS Information on their present peer review process • acceptance rate • peer review type • number of reviewers • instructions for peer reviewers • automated peer review tool • timeliness • international recommendations • ethical issues • etc. Opinions about open peer review • what is OPR? • opinions on possible improved quality, awards, acceptance, willingness of the reviewers to disclose their identity, bias, open discussion in the academic community, objectivity, etc. Open Science Fair 2017, 6-8 September, Athens, Greece
  12. 12. PEER REVIEW IN THE CROATIAN OA JOURNALS (1) • 58% employ peer reviewer from other institutions, 24% from the journal’s institution • 0% cascade peer review, 0% crowdsourcing peer review, 0% open peer review • most journals practice „double blind” peer review • 91% employ two or more peer reviewers per manuscript (mostly one Croatian, and one from abroad) • peer review process takes 120 days (mean) Open Science Fair 2017, 6-8 September, Athens, Greece
  13. 13. PEER REVIEW IN THE CROATIAN OA JOURNALS (2) • editors are mostly satisfied with the quality of the submitted reviews, and peer reviewers meet the deadlines • it is sometimes difficult to find competent peer reviewers • only 15% ask from the peer reviewers to declare conflict of interest • only 40% do plagiarism check before peer review • 45% have publicly available instructions for peer reviewers • 43% explain major ethical issues in the instructions Open Science Fair 2017, 6-8 September, Athens, Greece
  14. 14. OPEN PEER REVIEW – WHAT IS OPEN? Definition? Open Science Fair 2017, 6-8 September, Athens, Greece
  15. 15. Anthony Ross-Hellauer (OpenAIRE) - Open identities: Authors and reviewers are aware of each other’s identity. - Open reports: Review reports are published alongside the relevant article. - Open participation: The wider community to able to contribute to the review process. - Open interaction: Direct reciprocal discussion between author(s) and reviewers, and/or between reviewers, is allowed and encouraged. - Open pre-review manuscripts: Manuscripts are made immediately available (e.g., via pre-print servers like arXiv) in advance of any formal peer review procedures. - Open final-version commenting: Review or commenting on final “version of record” publications. - Open platforms: Review is de-coupled from publishing in that it is facilitated by a different organizational entity than the venue of publication. Open Science Fair 2017, 6-8 September, Athens, Greece
  16. 16. Görögh, Schmidt, Banelyte, Stanciauskas, Woutersen- Windhouwer: Practices, evaluation and mapping: Methods, tools and user needs (OpenUP) • Building upon OpenAIRE’s definition of OPR, the new peer review methods and tools will be examined in view of main attributes of openness: – open identity – open report – open participation – openness in time – open platform Open Science Fair 2017, 6-8 September, Athens, Greece
  17. 17. Layers of openness in peer review (1) 1.Who opens identity? Author, reviewer, member of the community, public 2. What is open? Comments, reviews, author's responses, submitted manuscript, different versions of the manuscript, final version of the accepted paper 3. Who is reviewing and who is commenting? Community, designated (formal) reviewers, public 4. What is commented? Manuscript, review, author responses, final version of the publication, everything Open Science Fair 2017, 6-8 September, Athens, Greece
  18. 18. Layers of openness in peer review (2) 5. When the review takes place? immediately after submission, during peer-review process, after the final version has been published 6. When is the manuscript/paper available? Immediately after submission, when the paper is accepted, after the paper was rejected 7. How is the communication process organized? discussions between reviewers, authors and reviewers, public and reviewers, reviewers and editors, authors and editors, moderated discussions? 8. Where the process of peer review takes place? journal, separate platform dedicated for peer review Open Science Fair 2017, 6-8 September, Athens, Greece
  19. 19. WHAT IS OPEN PEER REVIEW? Disclosed identity of the author(s) 11 9% Identities of both author and reviewer are disclosed 29 22% Review is publicly available 55 43% Disclosed identity of the peer reviewer 26 20% Author’s responses are publicly available 42 33% Former versions of the manuscript (before acceptance) are available 22 17% Post-print peer review 16 12% Readers can comment submitted and publicly available manuscripts 20 15% Don’t know 29 22% Open Science Fair 2017, 6-8 September, Athens, Greece
  20. 20. Open Science Fair 2017, 6-8 September, Athens, Greece 14% 10% 8% 10% 17% 12% 35% 26% 22% 33% 29% 29% 7% 18% 29% Open peer review would enhance peer review quality Open peer review would encourage discussions Open peer review would endanger objectivity 1 2 3 4 5
  21. 21. OPEN PEER REVIEW AND CROATIAN OA JOURNALS • not accepted in Croatian OA journals • some editors are not familiar with the concept of OPR • only 7% strongly agree that open peer review would enhance the peer review quality Open Science Fair 2017, 6-8 September, Athens, Greece

×