The Slides and Audio of the PeerJ presentation to the ICML (International Conference on Machine Learning) as part of their Workshop on Peer Reviewing and Publishing Models (June 20th, 2013), as organised by Andrew McCallum, University of Massachusetts, Amherst. See: https://sites.google.com/site/workshoponpeerreviewing/

1. Academic Publishing is Evolving…
Peer Review @
Pete Binfield
Co-Founder and Publisher
PeerJ
@ThePeerJ
https://peerj.com
@p_binfield
pete@peerj.com

2. Academic Publishing is Evolving…
(Some of) the Shortcomings of the Peer Review Process
- The peer review process is usually used to answer 2
unrelated questions: validity and interest.
- The decision of 2 anonymous reviewers determines the fate,
and possible reception, of an article
- The words and thoughts of reviewers are ‘lost’ to the ether
- Authors are unable to demonstrate the work and thought that
went into responding to reviewer comments
- Reviewers get no (tangible) reward
- Reviewers get no (attribution) credit
- There is little incentive to submit a review in a timely manner
- There is little incentive to take on a review in the 1st
place
- Unrealistic expectations are placed on pre-publication peer
review to validate a publication and ‘catch all the errors’.

5. Academic Publishing is Evolving…
PeerJ PrePrints
- A preprint server for the biological and medical sciences
- Preprint content is NOT peer reviewed
- Includes versioning functionality
- Engagement and commenting is linked to reputation metrics
PeerJ
- A broad based journal in the biological and medical sciences,
judging submissions based only on technical and scientific validity
- Fully peer reviewed, with rapid review process handled by a (very)
large editorial board of 800, including 5 Nobel Laureates
- Operates an optional ‘open peer review’ process
- Engagement and commenting is linked to ‘reputation metrics’
- Full suite of Article Level Metrics

6. Academic Publishing is Evolving…
The Reviewer’s Experience…
- Pre-publication reviews are formally invited
- Encouraged to provide their name to the authors
- Asked to comment only on scientific validity (in 3 categories of
‘Basic Reporting’, ‘Experimental Design’ and ‘Validity of the
Findings’)
- Choose from 4 Recommendations (Accept, Minor Revisions, Major
Revisions, Reject)
- User profiles are tied to Contribution credits
- Gain a tangible reward for providing on time reviews
Note: Reviewers / authors / commenters all use a ‘single sign on’

7. Academic Publishing is Evolving…
The Author’s Experience…
- Can test drafts, gain informal feedback, and post revisions to
PeerJ PrePrints
- Do not have to contort or distort their article to demonstrate
intangibles such as ‘novelty’, ‘broad interest’ or ‘high impact’
- Have the potential to see the names of their reviewers
- Are given the option to reproduce their peer review ‘audit trail’
on the published article
- Can recognize and reward insightful Feedback
- Accrue ‘alt-metrics’ from day of publication
- Are incentivized to participate in the peer review process

8. Academic Publishing is Evolving…
The Commenter’s Experience…
- Comments are framed as ‘feedback’
- Feedback is currently only available on PeerJ PrePrints
- No anonymous or pseudonymous commenting allowed
- User profiles are tied to Contribution points
The Reader’s Experience…
- Able to view the peer review process ‘in the raw’
- Can access ‘alt-metrics’ to help them form their own opinions
on any article
- Can provide Feedback and Comments and gain recognition
for doing so

9. https://peerj.com/articles/19/

12. https://peerj.com/articles/19/

13. https://peerj.com/macknik/

14. https://peerj.com/about/FAQ/academic-contribution/

15. https://peerj.com/articles/19/

16. https://peerj.com/reviews/48/

17. https://peerj.com/MathewWedel/

18. https://peerj.com/preprints/

19. https://peerj.com/preprints/8/

21. Academic Publishing is Evolving…
(Some of) the Shortcomings of the Peer Review Process
- The peer review process is usually used to answer 2
unrelated questions: validity and interest.
- The decision of 2 anonymous reviewers determines the fate,
and possible reception, of an article
- The words and thoughts of the reviewers are ‘lost’ to the
ether
- Authors are unable to demonstrate the work and thought that
went into responding to reviewer comments
- Reviewers get no (tangible) reward
- Reviewers get no (attribution) credit
- There is little incentive to submit a review in a timely manner
- There is little incentive to take on a review in the 1st
place
- Unrealistic expectations are placed on pre-publication peer
review to validate a publication and ‘catch all the errors’.

22. Academic Publishing is Evolving…
(Some of) the Shortcomings of the Peer Review Process
- The peer review process is usually used to answer 2
unrelated questions: validity and interest.
- The decision of 2 anonymous reviewers determines the fate,
and possible reception, of an article
- The words and thoughts of the reviewers are ‘lost’ to the
ether
- Authors are unable to demonstrate the work and thought that
went into responding to reviewer comments
- Reviewers get no (tangible) reward
- Reviewers get no (attribution) credit
- There is little incentive to submit a review in a timely manner
- There is little incentive to take on a review in the 1st
place
- Unrealistic expectations are placed on pre-publication peer
review to validate a publication and ‘catch all the errors’.
Objective Review Criteria
Objective Review Criteria + Alt Metrics
Open Peer Review
Open Peer Review
Free Membership for on-time reviews
Contribution Credit
Free Membership for on-time reviews
Un Peer-Reviewed PrePrints +
Open Peer Review +
Post Publication Feedback

23. Academic Publishing is Evolving…
Thank You
Pete Binfield
Co-Founder and Publisher
@p_binfield
pete@peerj.com
@ThePeerJ
https://peerj.com