Lecture 32: Open Peer Review Module 13 Dr. Jessica Laccetti
Outline Peer Review Open Peer Review Your Turn Homework
Peer Review Helps us find the “signal in the noise” (Leigh Dodds)
Peer Review Experts reviewing your paper/poster/chapter/book According to Dr. Vivian Siegel (Research Professor of Medicine and Cell and Developmental Biology at Vanderbilt University), journals employ peer review: to find out whether the conclusions drawn by the research are justified and “new” (i.e. whether you have a right to claim the discovery as “yours”) to get some gauge as to the significance or potential significance of the work to help decide whether your work is a match for the journal
Image from Peer Review in Scientific Publications: http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/856.pdf
Types of Peer Review Most common system = “single blind” peer review in which the author’s name and institution is known to the reviewer, but the reviewer’s name is not provided to the author. A number of journals instead choose to operate a “double blind” peer review system which is fully anonymised (i.e. the author(s) are unaware of the identity of the reviewer(s) and vice versa). Recently, there have been some experiments with a third type, “open” peer review, in which the authors’ and reviewers’ names are revealed to each other. […] Open peer review can be reasonably described as an experimental system at this stage and is far from common. http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/856.pdf
Draw-Backs of Traditional Peer Review The traditional ways of scientific publishing and peer review do not live up to the needs of efficient communication and quality assurance in today’s highly diverse and rapidly developing world of science.
Advantages of Open Peer Review Open access is fully compatible with traditional peer review, and beyond that it enables interactive and transparent forms of review and discussion open to all interested members of the scientific com- munity and the public (interactive open access peer review alias public, collaborative or community peer review). Ref: 294 Liber Quarterly Volume 19 Issue 3/4 2010
Advantages of Open Peer Review Open access gives reviewers more information to work with, i.e., it provides unlimited access to relevant publications across different scientific disciplines and communities (interdisciplinary scientific discussion and quality assurance). Ref: 294 Liber Quarterly Volume 19 Issue 3/4 2010
Advantages of Open Peer Review Open access facilitates the development and implementation of new metrics for the impact and quality of scientific publications (combination of citation, download/usage, commenting and ranking by various groups of readers and users, respectively). Open access helps to overcome the obsolete monopoly/oligopoly structures of citation counting which have been stifling innovation in scientific research, publishing and evaluation over decades. Ref: 294 Liber Quarterly Volume 19 Issue 3/4 2010
Advantages of Open Peer Review The possibility of comparing a final revised paper with the preceding discussion paper and following the interactive peer review and public discussion also facilitates the evaluation of individual publications for non-specialist readers and evaluators. The style and quality of interactive commenting and argumentation provide insights that go beyond, and complement, the information contained in the research article itself.
Findings of U.K. Parliament Report We conclude that different types of peer review are suitable to different disciplines and research communities. We consider that publishers should ensure that the communities they serve are satisfied with their choice of peer-review methodology. Publishers should keep them updated on new developments and help them experiment with different systems they feel may be beneficial.http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/856.pdf
Recommendations of U.K. Parliament Report The principles of openness and transparency in open peer review are attractive, and it is clear that there is an increasing range of possibilities. There are mixed results in terms of acceptance amongst researchers and publishers, although some researchers are keen to see greater transparency in their fields. We encourage publishers to experiment with the various models of open peer review and transparency and actively engage researchers in taking part. http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/856.pdf
PLoS The approach adopted by PLoS ONE—where the peer review process focuses solely on whether the findings and conclusions are justified by the results and methodology presented, rather than on assessment of the relative importance of the research or perceived level of interest it will generate—has both reduced the burden on the reviewer and the time it takes to get a paper published
Your Turn In small groups and in a google doc that is public and editable: Respond to the aforementioned conclusions found by the U.K. Parliamentary Report Think about what kind of peer review would be best suited to your own field of study (blind, double-blind, open, closed) How might you improve the quality of peer review? Post a link to your PUBLIC google doc on the blog Pick a link to someone else’s response and add some comments in the form of an academic peer review
Homework Online class! By the end of class (9:50) please complete the following exercise: As the new editor–in–chief of a significant journal published by Reed Elsevier you would like to modernize the academic publishing process. You are eager to implement “open peer review” See http://en.wikipedia.org/wiki/Open_peer_review even after Nature’s experiment with this form of peer review failed (see http://www.nature.com/nature/peerreview/debate/nature05535.html). However, a more recent experiment by Noah Wardrip-Fruin on the Grand Text Auto blog was more successful (see http://grandtextauto.org/2009/05/12/blog- based-peer-review-four-surprises/). Do you try to convince your colleagues to try open peer review or are you daunted by examples such as Nature’s? Upload your 3-5-paragraph response to Google docs. Make sure you share your document so it is visible to anyone and add a link to the document as a comment on the Lecture 33 post. Be sure to e-mail the link to me and your TA as well.