Notwithstanding its significance in the process of academic publishing, peer review generally has a bad press, sometimes due to unrealistic expectations of what peer review should achieve. Some argue that the model of traditional peer review is now moribund, and certainly the advent of new technologies has shaped recent developments in the process of peer review. This session will explore where peer review has come from, where it is going, and what future it has.

1. ‘Peer review is dead: long live peer review!’
Michael Willis
Wiley, Oxford, UK
miwillis@wiley.com
@ctyerkes
1

2. A venerable legacy
2
"Philosophical Transactions Volume 1 frontispiece" by Henry Oldenburg - Philosophical Transactions. Licensed under CC BY 4.0 via Wikimedia
Commons - http://commons.wikimedia.org/wiki/File:Philosophical_Transactions_Volume_1_frontispiece.jpg

3. Augustus Bozzi Granville" by Unknown - Wellcome Images. Licensed under Public Domain via Wikimedia Commons -
http://commons.wikimedia.org/wiki/File:Augustus_Bozzi_Granville.jpg#mediaviewer/File:Augustus_Bozzi_Granville.jpg
A venerable tradition
3
• Augustus Bozzi Granville (1783-
1872)
• First medical autopsy on an
Ancient Egyptian Mummy,
described to the Royal Society in
1825
• Critique of peer review as
conducted by Royal Society

4. Science without a head; or, the Royal Society dissected
London, 1830
The Royal Society in the 19th
century; being a statistical
summary of its labours during the last thirty-five years, with
many original tables and official documents (never before
published)
London, 1836
Polemic against the old order
4

5. Peer review is…
Arbitrary
‘Much oftener is the fate of a paper committed to the chances of
the mere yea-and-nay box, than to the decision of a competent
judge instructed to offer a preliminary opinion upon its merits.’
Incompetent
‘At many of these meetings, members of the committee of papers
have been present who have not the smallest pretension to any
knowledge whatever of the subject under consideration.’
Critique of method
5

6. ‘It has been said that the warp that holds the complex
fabric of science together is peer review, and the woof
is the noise made by scientists who complain about it.’
Jukes, TH. Peer review. Nature 265 (1977); 203
‘Those who complain of unfair treatment at the hands
of reviewers would do well to remember that 400
years ago they would have had—at best—the choice
between the burning of their manuscripts or of
themselves.’
Grivell, L. Through a glass darkly: The present and the future of
editorial peer review. EMBO reports 7 (2006); 567-570
Leaping forward…
6

7. ‘…slow, expensive, profligate of academic
time, highly subjective, prone to bias, easily
abused, poor at detecting gross defects, and
almost useless for detecting fraud…’
Smith, R. Peer review: reform or revolution? BMJ 315 (1997): 759–760
A damning indictment
7

8. Nails in the coffin of peer review?
8
It’s a
black box
It’s a
black box
There are too
few reviewers
to do the work
There are too
few reviewers
to do the work
It takes
too long
It takes
too long
It costs
too much
It costs
too much
Microsoft Clip ArtIt’s not
rigorous
enough
It’s not
rigorous
enough
The web
changes
everything
The web
changes
everything

9. How strong are the nails?
9

10. Why does this matter?
furthering of
scientific
endeavour
health and welfare
of society
public and
academic
confidence in
scientific record
integrity of
scientific record
10

11. ‘Many forms of Government have been tried, and
will be tried in this world of sin and woe. No-one
pretends that democracy is perfect or all-wise.
Indeed, it has been said that democracy is the
worst form of Government except all those other
forms that have been tried from time to time.’
Peer review resurgens
11
peer reviewpeer review
traditional peer
review
traditional peer
review
traditional peer
review
traditional peer
review
peer reviewpeer review
"Sir Winston S Churchill" by United Nations Information Office,
New York - Library of Congress, Reproduction number LC-
USW33-019093-C
© US Federal Government

12. Peer review: alive and kicking
12
clipartpanda.com
‘…peer review is not only alive and kicking, but
apparently increasing its influence, despite the
many potential (or invented) threats posed by a
rapidly unfolding and enveloping digital
environment…’
Peer review: still king in the digital age, Nicholas et al.,
Learned Publishing 28(2015):15-21

13. Peer review under increased scrutiny
13

14. —Research indicates that the vast majority of
researchers still value and trust peer review
—More scholars are looking at peer review as a
scientific object of study
—There are now more organisations devoted to
understanding and improving peer review than
ever before
—There are several new products and services
designed to preserve and enhance peer review
‘Four reasons to feel good about the
future of peer review’
14
Eric Hall
http://www.researchinformation.info/news/news_story.php?news_id=1862
10 March 2015

15. Peer review in scientific publications (July 2011)
http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/856.pdf
‘…despite the many criticisms and the little solid evidence on its efficacy,
editorial peer review is considered by many as important and not
something that can be dispensed with.’
‘In order for current peer-review practices to be optimised and innovative
approaches introduced, publishers, research funders and the users of
research outputs (such as industry and government) must work
together.’
‘…we encourage the prudent use of online tools for post-publication
review and commentary as a means of supplementing pre-publication
review.’
‘The integrity of the peer-review process can only ever be as robust as the
integrity of the people involved.’
UK Parliament on peer review
15

16. Peer review survey, Sense About Science (2009)
http://www.senseaboutscience.org/data/files/Peer_Review/Peer_Review_Survey_Fi
nal_3.pdf
—What do you think of peer review overall?
—What value does peer review add to
research output?
—How can peer review be improved?
Researchers on peer review
16

17. Researchers on peer review
What do you think of peer review overall?
17

18. Researchers on peer review
What value does peer review add?
18

19. Researchers on peer review
How can peer review be improved?
19

20. Joynson C and Leyser O 2015 [v1; ref status: approved 1, http://f1000r.es/53j] F1000Research
2015, 4:66 (doi: 10.12688/f1000research.6163.1)
The culture of scientific research, Joynson C and Leyser O.
Conclusions of 2014 survey by Nuffield Council on Bioethics

21. Researchers on peer review
Training: an opportunity for publishers
21

22. Find out
what others
are doing in
your area of
research
Critique the
research
undertaken
by others
Review
manuscripts
in your area
of research
Inform your
own research
Conduct your
own research
The virtuous circle
of peer review and
research community
engagement

23. Reviewer accountability 1:
A creditable activity
23Spectrum (Official newsletter of the Australian Institute of Radiography), November 2014

24. Reviewer accountability 2:
Public recognition
24
showcaseshowcase
get creditget credit

25. Reviewer accountability 3:
The Hippocratic Oath of peer review
25
transparencytransparency reproducibilityreproducibility

26. Common themes for authors
26
transparencytransparency
rigourrigour
speedspeed
collaborationcollaboration

27. ‘Aim high, publish fast’
27

28. Peer review variables
Transparency
Timing
Submission
process
Assessment
of…
Reviewer
selection
Decision
process
-Open peer review
-Closed peer review
-Pre-publication
-Post-publication
-Direct to a journal
-To a 3rd
party
-Transfer within “family”
-Science only
-Science + novelty or impact
-Independent review
-Interactive review
-Collaborative review
-Authors recruit reviewers
-Editors recruit reviewers
-Reviewers “bid” for papers

29. ‘…publishers would welcome more
guidance from key sections of the
research community on the kinds of peer
review services they want publishers to
provide, and on the purposes that they
should seek to fulfil.’
Scholarly Communication and Peer Review: The Current
Landscape and Future Trends, Research Information Network
CIC, March 2015
Peer review evolution:
A collaborative effort
29

30. The journey of peer review evolution
30
Listen to the community
Work with the community
Do more research
Be flexible
Train, educate, evaluate
Collaborate with
stakeholders

31. Collaborate with stakeholders
31

32. 32