Your SlideShare is downloading. ×
0
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Peer review
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Peer review

185

Published on

This is a copy of a talk i gave at the Vision Sciences Society meeting in 2012. The talk presentas a review of the history and evolution of peer review and scientific publishing, an evaluation of its …

This is a copy of a talk i gave at the Vision Sciences Society meeting in 2012. The talk presentas a review of the history and evolution of peer review and scientific publishing, an evaluation of its current function based on survey data, and a proposal for improving the system.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
185
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. The state of scientific publishing and a new model Dwight Kravitz Chris Baker
  • 2. Motivation: Improving Cost and Quality NIH Budget / # Papers Budget per Paper ($1000) Published
  • 3. Peer Review: Outline• History and evolution• Modern purpose• Improving peer review• Post-publication
  • 4. History of Peer Review and Publishing• Problems facing authors – Disseminating results – Claiming ownership of results• Publishing – Publishers had distribution channels – Provides a record of when results were
  • 5. History of Peer Review and Publishing• Problems facing publishers – Establishing veracity of results – Deciding which results to publish given limited• Peer review space – Stamp of scientific approval – Prioritizes papers for Henry Oldenburg, 1665 inclusion by publishers Medical Essays and Observations, 1731
  • 6. 55% Reject42% Revise3% Accept
  • 7. 55% Reject42% Revise3% Accept
  • 8. Given Revision: 98% Accept 2% Reject
  • 9. 64% of papers are rejected at least once
  • 10. Direct Costs of the Current System• Total Number of Reviews: 6.3 (2-15)• Total Days under Review: 122 (21-321)• Total Hours for Revisions: 68 (5-300)• Total Days to Publish: 221 (21- 533)• Publication Fees: $2000• Thousands of dollars and hundreds of hours per paper• Long delays impair every form of assessment• Variance hinders planning research programs and careers
  • 11. Peer Review: Outline• History and evolution – Invention – Current form• Modern purpose• Improving peer review• Post-publication
  • 12. Original Purpose• Reasons for publishing – Disseminating results – Claiming ownership of results• Reasons for peer review – Establishing veracity of results – Deciding which results to publish given limited space Henry Oldenburg, 1665 Medical Essays and Observations, 1731
  • 13. Modern Purpose• Reasons for publishing – Disseminating results – Claiming ownership of results• Reasons for peer review – Establishing veracity of results Abstract: 9495 – Deciding which results to Full Text: 8564 publish given limited PDF: 2549 space
  • 14. Pre-reception• Peer review allows authors to test the paper before sending it to the whole field – Catches fundamental errors – Highlights important overlooked issues
  • 15. Problems with Pre-reception• Reviews are impure measures of quality
  • 16. Problems with Pre-reception• Reviews are impure measures of quality• Review process is opaque to the wider field
  • 17. Prioritization of the Literature• Loop ranks papers based on which journal publishes them – Provides a way of deciding which papers to read – Compensates for long publication lags
  • 18. Problems with Prioritization• Reviews not focused solely on quality• Inefficient (avg. lag is 221 days)• Creates detrimental short-term incentives
  • 19. Problems with Prioritization Budget per Paper ($1000) http://pmretract.heroku.com
  • 20. Problems with Prioritization Budget per Paper ($1000) Retractions per 100k Papers http://pmretract.heroku.com
  • 21. Problems with Prioritization Fang &Casadevall, 2011
  • 22. Problems with Prioritization• Reviews not focused on solely on quality• Inefficient (avg. lag is 221 days)• Creates detrimental short-term incentives• Doesn’t work
  • 23. Problems with Prioritization
  • 24. Problems with Prioritization
  • 25. Problems with Prioritization
  • 26. Problems with Prioritization
  • 27. Peer Review: Outline• History and evolution – Invention – Current form• Modern purpose – Pre-reception – Prioritization• Improving peer review• Post-publication
  • 28. Improving the current system• Make publishing more efficient and less variable • Days to Publish: 221 (21- 533) • Years to First Cite: ~1.5 + research• Capture reviews in the prioritization • Total Number of Reviews: 6.3 • Total Hours of Reviewing: ~31.5Decouple peer review rather than• Focus reviews on science from publishing
  • 29. Proposed System
  • 30. Decoupling peer review from publishing• Allows reviewers to focus on scientific merit rather than publication• Allows reviews to be captured for prioritization and rewards for reviewers• Makes publishing simpler and more efficient• Eliminates the short-term incentive to produce least publishable units
  • 31. Peer Review: Outline• History and evolution – Invention – Current form• Modern purpose – Pre-reception – Prioritization• Improving peer review – Decoupling from publishing• Post-publication
  • 32. Format for papers
  • 33. The need for post-publication review Proportion of field Why Shru Useful Field ? g Datapoint Altering
  • 34. The need for post-publication review We need more reviews to Proportion of field approximate these complex distributions. Also enables personalized prioritization of the literature. Why Shru Useful Field ? g Datapoint Altering
  • 35. Incentivizing post-publication review
  • 36. Organizing the literature• The utility of modern peer review and publishing is prioritizing and organizing the literature.
  • 37. Organizing the literature• The utility of modern peer review and publishing is prioritizing and organizing the literature.
  • 38. Peer Review: Outline• History and evolution – Invention – Current form• Modern purpose – Pre-reception – Prioritization• Improving peer review – Decoupling from publishing• Post-publication review

×