Notwithstanding its significance in the process of academic publishing, peer review generally has a bad press, sometimes due to unrealistic expectations of what peer review should achieve. Some argue that the model of traditional peer review is now moribund, and certainly the advent of new technologies has shaped recent developments in the process of peer review. This session will explore where peer review has come from, where it is going, and what future it has.
UKSG Conference 2015 - “Peer review is dead, long live peer review.” Michael Willis Wiley
1. ‘Peer review is dead: long live peer review!’
Michael Willis
Wiley, Oxford, UK
miwillis@wiley.com
@ctyerkes
1
2. A venerable legacy
2
"Philosophical Transactions Volume 1 frontispiece" by Henry Oldenburg - Philosophical Transactions. Licensed under CC BY 4.0 via Wikimedia
Commons - http://commons.wikimedia.org/wiki/File:Philosophical_Transactions_Volume_1_frontispiece.jpg
3. Augustus Bozzi Granville" by Unknown - Wellcome Images. Licensed under Public Domain via Wikimedia Commons -
http://commons.wikimedia.org/wiki/File:Augustus_Bozzi_Granville.jpg#mediaviewer/File:Augustus_Bozzi_Granville.jpg
A venerable tradition
3
• Augustus Bozzi Granville (1783-
1872)
• First medical autopsy on an
Ancient Egyptian Mummy,
described to the Royal Society in
1825
• Critique of peer review as
conducted by Royal Society
4. Science without a head; or, the Royal Society dissected
London, 1830
The Royal Society in the 19th
century; being a statistical
summary of its labours during the last thirty-five years, with
many original tables and official documents (never before
published)
London, 1836
Polemic against the old order
4
5. Peer review is…
Arbitrary
‘Much oftener is the fate of a paper committed to the chances of
the mere yea-and-nay box, than to the decision of a competent
judge instructed to offer a preliminary opinion upon its merits.’
Incompetent
‘At many of these meetings, members of the committee of papers
have been present who have not the smallest pretension to any
knowledge whatever of the subject under consideration.’
Critique of method
5
6. ‘It has been said that the warp that holds the complex
fabric of science together is peer review, and the woof
is the noise made by scientists who complain about it.’
Jukes, TH. Peer review. Nature 265 (1977); 203
‘Those who complain of unfair treatment at the hands
of reviewers would do well to remember that 400
years ago they would have had—at best—the choice
between the burning of their manuscripts or of
themselves.’
Grivell, L. Through a glass darkly: The present and the future of
editorial peer review. EMBO reports 7 (2006); 567-570
Leaping forward…
6
7. ‘…slow, expensive, profligate of academic
time, highly subjective, prone to bias, easily
abused, poor at detecting gross defects, and
almost useless for detecting fraud…’
Smith, R. Peer review: reform or revolution? BMJ 315 (1997): 759–760
A damning indictment
7
8. Nails in the coffin of peer review?
8
It’s a
black box
It’s a
black box
There are too
few reviewers
to do the work
There are too
few reviewers
to do the work
It takes
too long
It takes
too long
It costs
too much
It costs
too much
Microsoft Clip ArtIt’s not
rigorous
enough
It’s not
rigorous
enough
The web
changes
everything
The web
changes
everything
10. Why does this matter?
furthering of
scientific
endeavour
health and welfare
of society
public and
academic
confidence in
scientific record
integrity of
scientific record
10
12. Peer review: alive and kicking
12
clipartpanda.com
‘…peer review is not only alive and kicking, but
apparently increasing its influence, despite the
many potential (or invented) threats posed by a
rapidly unfolding and enveloping digital
environment…’
Peer review: still king in the digital age, Nicholas et al.,
Learned Publishing 28(2015):15-21
14. —Research indicates that the vast majority of
researchers still value and trust peer review
—More scholars are looking at peer review as a
scientific object of study
—There are now more organisations devoted to
understanding and improving peer review than
ever before
—There are several new products and services
designed to preserve and enhance peer review
‘Four reasons to feel good about the
future of peer review’
14
Eric Hall
http://www.researchinformation.info/news/news_story.php?news_id=1862
10 March 2015
15. Peer review in scientific publications (July 2011)
http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/856.pdf
‘…despite the many criticisms and the little solid evidence on its efficacy,
editorial peer review is considered by many as important and not
something that can be dispensed with.’
‘In order for current peer-review practices to be optimised and innovative
approaches introduced, publishers, research funders and the users of
research outputs (such as industry and government) must work
together.’
‘…we encourage the prudent use of online tools for post-publication
review and commentary as a means of supplementing pre-publication
review.’
‘The integrity of the peer-review process can only ever be as robust as the
integrity of the people involved.’
UK Parliament on peer review
15
16. Peer review survey, Sense About Science (2009)
http://www.senseaboutscience.org/data/files/Peer_Review/Peer_Review_Survey_Fi
nal_3.pdf
—What do you think of peer review overall?
—What value does peer review add to
research output?
—How can peer review be improved?
Researchers on peer review
16
20. Joynson C and Leyser O 2015 [v1; ref status: approved 1, http://f1000r.es/53j] F1000Research
2015, 4:66 (doi: 10.12688/f1000research.6163.1)
The culture of scientific research, Joynson C and Leyser O.
Conclusions of 2014 survey by Nuffield Council on Bioethics
22. Find out
what others
are doing in
your area of
research
Critique the
research
undertaken
by others
Review
manuscripts
in your area
of research
Inform your
own research
Conduct your
own research
The virtuous circle
of peer review and
research community
engagement
23. Reviewer accountability 1:
A creditable activity
23Spectrum (Official newsletter of the Australian Institute of Radiography), November 2014
29. ‘…publishers would welcome more
guidance from key sections of the
research community on the kinds of peer
review services they want publishers to
provide, and on the purposes that they
should seek to fulfil.’
Scholarly Communication and Peer Review: The Current
Landscape and Future Trends, Research Information Network
CIC, March 2015
Peer review evolution:
A collaborative effort
29
30. The journey of peer review evolution
30
Listen to the community
Work with the community
Do more research
Be flexible
Train, educate, evaluate
Collaborate with
stakeholders
Figure 1. Suggestions for action to support good research practice and the production of high quality science.
How to cite: Joynson C and Leyser O 2015 The culture of scientific research [v1; ref status: approved 1, http://f1000r.es/53j] F1000Research 2015, 4:66 (doi: 10.12688/f1000research.6163.1)