Emerging open peer review journals in scientific publishing, transparency in peer review ...... https://www.ideals.illinois.edu/handle/2142/89432

1. Peiling Wang1, Manasa Rath1, Michael Deike1, Qiang Wu2
1University of Tennessee at Knoxville, USA
2University of Science and Technology, China
Open Post-Publication Peer Review: An
Innovation in Scientific Publishing

2. Peer Review: Bedrock of Modern Science
 Evaluation of scientific findings
 Quality assurance for scholarly or scientific
publications
 Criticisms:
 long delay in publishing
 high quality work not recognized thus rejected
 flawed research published
o a scientist published 126 fabricated papers
o “peer review ring” scandal

3. Globalization of Modern Science
 Big science
 volume (e.g., 2006 1,346,000 articles published in
23,750 peer-reviewed journals)
 growth (e.g., > 3.31% for refereed journals)
 increased collaboration: number of co-authors;
number of participating countries
 Prolific authors
 Hirsch index ~100
 Increased accessibility to publications and
data

4. Open Peer Review: Much Needed
Modernization
 Peer review is an established practice of
scientific publishing for centuries
 Blind (double or single) referee process
for objectivity may compromise
transparency
 Open science and open research
 Open access
 Open peer review
 Open assessments (reader comments)

5. Open Science & Open Research
 Open Access (OA)
 Open Assessment
 F1000Prime (peer recommendation)
 Reader comments (open or registered users)
 Open Peer Review (OPR)
 Process transparent
 Identity known (author, or/and referee)
 Review history (published alongside)

6. Two Basic OPR Models
 Post-publication open peer review
 Articles published online upon submission
 Peers are invited to review
 Signed review report immediately online
 Must pass Indexation threshold
 Pre-publication open peer review
 Blind (non blind) review moderated by editors
 Publish review history alongside accepted
article (democratic decisions)

8. Indexation vs. Publication Timelag
If an article is published before peer review,
what is the quality assurance?
A threshold is set for indexing a post-
publication article:
 Two referees assigned “Approval”
or
 One referee assigned “Approval” plus two
referees assigned “Approval with reservation”

9. OPR: Innovation
 New and less known by scientists who
publish or review
 Review history – never before published
as a part of the article
 Authors and reviewers become
collaborators in open research
 Future of OPR is still in the making

10. Research Method
 Comparing four OPR journals
 Timelag (publication vs. indexation)
 Openness policy (review history, anonymity,
…)
 Analyzing F1000Research (as the post-
publication peer review model)
 focus on only research articles

11. Timelags
OPR Journals Timelag (Median)
F1000Research Publication: ~7 days Indexation: 35 / 69 days
PeerJ Acceptance: 74 days Publication: 22 days
eLife Acceptance: 90 days Publication: 29 days
GigaScience
Acceptance: 102
days
Publication: 22 days

12. OPR Policies
PeerJ (Pre-publication)
Traditional process
(blind peer review)
Anonymity:
reviewer’s decision
Peer Review history:
author’s decision
Author response
F1000 (Post-publication)
New process (totally
open process)
Non-anonymous OPR
policy
Immediate online
review reports
Immediate author
responses

13. OPR Policies (Cont’d)
eLife
Decision letter
published (Editor)
Reviewer identity
(optional)
Author’s response
GigaScience
Notes on datasets
Non-anonymity
(may honor
reviewer’s request)
Review history
Author’s responses

14. F1000Research: A Fully Open Model
 F1000Research dataset, harvested on
March 20, 2015, 215 indexed research
articles
 58% were approved in their first version
and being indexed in Scopus & PubMed
 For the articles approved in their original
submissions, 21% published a revised
version

15. F1000Research Articles (215 indexed)
 929 authors from 50 countries
 544 referees from 46 countries
 Top 5 contributing countries:
 Authors from
US, UK, Australia, Germany, Canada
 Reviewers from
US, UK, Canada, Australia, Germany

16. Where else do these authors publish?
Searching WoS and Scopus for the top
downloaded F1000Research articles’ 96
authors:
 The authors’ H-index (mean 11; the highest H-
index 97)
 Journals’ high impact factor (mean 9.7; the
highest IF 42.3)
 Journals are reputable, such as Science, Nature,
and Annual Review of Entomology

17. Advantages of OPR
 transparency
 openness
 review history as an integral part of
published research work
 accountability of both authors and
reviewers to their work
 open research to involve readers in
evaluation of published work

18. Further Research on OPR
 Investigate the state-of-the-art of OPR
 Identify the best practice
 Investigate the effect of OPR on referees
 What are the differences between the signed
review reports and anonymous review
reports?
 Investigate the effect of OPR on authors
 Willingness to adopt OPR

19. Dual Models: Open or Traditional
Papers in Physics: % of Articles under OPR
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
2009 2010 2011 2012 2013 2014 2015 2016