Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Accept with revisions: The evolving peer review landscape

378 views

Published on

An overview of the current state of peer review, how the community is addressing challenges in peer review quality, and the future outlook of peer review "with revisions."

Published in: Education
  • Login to see the comments

  • Be the first to like this

Accept with revisions: The evolving peer review landscape

  1. 1. Accept with revisions: The evolving peer review landscape Danielle Padula, Scholastica Community Development
  2. 2. Circa 1940: Peer review is the bedrock of quality research… but not without its challenges “Off to a Patchy Start: Milestones in Journal Peer Review Research,” article and images by Hilda Bastian
  3. 3. Scholastica Perspective: Peer review hub for editors, authors, and reviewers
  4. 4. ❏ The state of peer review ❏ How the industry is addressing key challenges ❏ Future outlook: peer review “with revisions” What we’ll cover
  5. 5. “ More than half of the journals accepted the paper, failing to notice its fatal flaws. Beyond that headline result, the data from this sting operation reveal the contours of an emerging Wild West in academic publishing. - John Bohannon, Who's Afraid of Peer Review?
  6. 6.
  7. 7. The replication crisis
  8. 8. Peer review is broken. ...or is it?
  9. 9. We need a diagnosis Is peer review upholding scholarly ideals/norms?
  10. 10. Researchers want to improve not replace peer review Publons Global State of Peer Review report (2018) ◎ 66.8% say peer review is extremely important ◎ 84.8% say institutions should better recognize contributions Sense About Science Peer Review Survey (2009) ◎ Only 32% say peer review is best it can be ◎ 79% say peer review should identify best papers, determine originality/ importance, spot fraud
  11. 11. Key questions for editors, authors, and reviewers ◎ Is research reproducibility possible and encouraged? ◎ Are journals operating transparently/following core standards? ◎ Is equitable access to opportunities and diversity in scholarship apparent and encouraged?
  12. 12. “ The question shouldn’t be whether the process is perfect. Rather, as with all scientific processes, does this process yield a better result than might otherwise be expected without it. - Todd Carpenter, Ask The Chefs: Peer Review Quality (2019)
  13. 13. Triaging peer review Time for some revisions
  14. 14. Systemic problems in the research culture to address “Mertonian” scientific norms/ideals (CUDOS) ◎ Communality ◎ Universalism ◎ Disinterestedness ◎ Organized skepticism Scientific perceptions (Anderson, Martinson, & DeVries, 2007) ◎ Secrecy ◎ Particularism ◎ Self-interestedness ◎ Organized dogmatism
  15. 15. “Mertonian” scientific norms/ideals (CUDOS) ◎ Communality ◎ Universalism ◎ Disinterestedness ◎ Organized skepticism Scientific perceptions (Anderson, Martinson, & DeVries, 2007) ◎ Secrecy ◎ Particularism ◎ Self-interestedness ◎ Organized dogmatism
  16. 16. Reproducibility Is research reproducibility possible and encouraged?
  17. 17. Registered Reports: Peer review before results Center for Open Science: https://cos.io/rr/
  18. 18. “ Registered Reports eliminates the bias against negative results in publishing because the results are not known at the time of review. - Daniel Simons, Professor at U Illinois, Chief Editor of Advances in Methods and Practices in Psychological Science
  19. 19. Open science badges to encourage reproducibility
  20. 20. Open science badges to encourage reproducibility
  21. 21. Transparency and standards Are journals operating transparently/following core standards?
  22. 22. TOP Guidelines from the Center for Open Science
  23. 23. Duty of publishers, funders, non-profits: Establish journal processes and standards
  24. 24. Duty of authors & institutions: Identify trusted journals and follow research standards https://thinkchecksubmit.org
  25. 25. Preprints as a model for transparency pre- and post-publication ◎ Green open access ◎ Preprint + open peer review ◎ Preprint overlay publishing model ○ E.g. Discrete Analysis
  26. 26. “ We encourage authors to post updated versions on the arXiv if they discover ways in which their papers can be improved. If notified about these, we will draw attention to them on the journal website. - Discrete Analysis, For Authors page
  27. 27. MECA: Manuscript Exchange Common Approach Transfering manuscripts between journals/systems: ◎ Easier to submit manuscript to different journal ◎ Reduce frequency of repeat reviews
  28. 28. Is equitable access to opportunities and diversity in scholarship apparent and encouraged? Diversity and inclusion
  29. 29. Launch of the Coalition for Diversity and Inclusion in Scholarly Communications (C4DISC) ◎ Statement of Principles ◎ Market research/analysis ◎ Best practices and training resources ◎ Outreach programs
  30. 30. “ The value of the Coalition is being able to share that information across groups, thereby increasing dissemination and impact. Forming the Coalition sends the message [...] that as an industry, we’re committed to addressing these issues. - Melanie Dolechek, Executive Director of SSP interview
  31. 31. 1. Focus on making journal articles widely accessible 2. Low or no publication fees 3. Track data around diversity and make an action plan 4. Take steps to support ESL authors 5. Make diversity and inclusion part of the journal mission Steps towards greater diversity and inclusion blog.scholasticahq.com/post/ways-academic-journals-can-increase-diversity-peer-review/
  32. 32. ❏ The state of peer review ❏ How the industry is addressing key challenges ❏ Future outlook: peer review “with revisions” Where we’ve been and where we’re going: Monitoring the health of peer review
  33. 33. Additional resources for journal editors https://scholasticahq.com/resources
  34. 34. THANKS! @scholasticahq support@scholasticahq.com

×