Is Peer Review Peerless? Tony Eklof James Joyce Library University College Dublin LIR Annual Seminar 2009
Peer Review• What it is• The peer review process• Why peer review• Criticisms• Peer review in the sciences• Peer review in the humanities• Conclusions
The Frescoes of Domenico Ghirlandaio:A Study in High Renaissance and Early Mannerist Florence
Journal peer review process Paper is accepted for publication, Paper to EditorReturned with suggestions for improvements, Editor to Experts in Or rejected. subject area (Peers) (resubmitted elsewhere) Peers consider for validity, Significance and originality.
Double Blind Review• Both authors & referees anonymous• Survey shows it is preferred• Support highest in humanities and amongst female authors (reduces bias)• Causes some problems with reviewers ie comparing with earlier works• At odds with the open sharing of information and transparency?
Open Review• Some researchers now post pre- publication versions on web to invite feedback before formal submission• Increasingly reader’s comments and criticisms, particularly for open access journals, add a positive element to the process
Why Peer Review?• Quality control for scholarly information• Weeds out fraud• Lessens workload of Editor• Promotes originality and academic rigour – Mechanism for improvement of manuscripts• Adds a ‘human judgement’ element to the academic process
Criticisms• Slows up the research process• Time consuming for reviewers• For some manuscripts or proposals it may be difficult to find experts qualified to review• Bias of reviewer• Arbitrary, secret and subjective!• Researchers can be frustrated by process, valuable time spent on ‘lateral’ research
Nature’s Peer Review Trial• Open Peer Review• 1 June-30 September 2006• Authors of new papers submitted invited to have papers hosted on internet for public comment• Papers simultaneously subjected to standard peer review
Nature’s Open Review• 71 out of 1,369 authors agreed to display papers for open comment (5%)• Healthy online traffic, but did not convert into significant amount of useful comment• Some authors reluctant for fear of ideas being ‘scooped’• Nature to continue to explore open review but not to implement system
Peer Review in Humanities• Applying bibliometric indicators in Arts can be problematical• Very broad range of subjects• Some esoteric and specialised• Interdisciplinary nature• Performing arts difficult to peer review
Conclusion• Peer review is a flawed, much criticised system of insuring academic rigour in published journal articles• There is no better alternative system on offer• System improved by open and or double blind review, and by training for peer reviewers