2. Peer review at eLife
• ~30% of submissions sent for full peer review
• All reviewers see all reviews (and identities) and the
Reviewing Editor leads a discussion
– i) Ask for revision (~60%): list of essential revisions sent to
author (ie, not full reviews). Only ask for extra work if it is
needed to support main conclusions and can be done in
about two months
– ii) Reject (~40%)
• Reviewing Editor usually makes decision on revision
• Publish decision letter and author response
• Aim is to be fast, fair and transparent
3. Study 1: Author-reviewer homophily in
peer review
• Murray, Siler, Larivière, Chan, Collings, Raymond,
Sugimoto. 2019. bioRxiv: 400515
• 28876 initial submissions and 7192 full submissions
• Outcomes were more favourable for male authors
– Acceptance rate for male corresponding authors = 15.6%
– Acceptance rate for female corresponding authors = 13.8%
– Similar bias seen for last authors but no difference between male
and female first authors
4. Study 1: Author-reviewer homophily in
peer review
• Outcomes were more favourable for authors affiliated with
institutions in North America and Europe
• Both groups were over-represented among gatekeepers.
• Outcomes were influenced by homophily
– When all reviewers were male, accept rate was 55.9% for male
last authors and 51.1% for female last authors
– For mixed-gender review teams the disparity was smaller and not
significant
– Last author-reviewer homogeny had accept rate of 57.4%; non-
homogeny had accept rate of 47.4%
5. Study 2: Results from a peer-review trial
• For papers sent for peer review as part of trial, authors
could decide how they responded to the reviews
• Published paper would include assessment from Editor
• 313 papers in trial; 612 regular papers; not randomized
• Data still being analyzed; initial findings reported in two
blog posts on: elifesciences.org/inside-elife
• Encourage rate = 22.4% (cf 26.4% for regular papers)
• Results suggest different decision outcomes based on
career stage.
6. What are we doing based on the results
of these studies?
• Working to improve the diversity of our editorial boards
– BRE is 68% male/32% female
– Aiming for 60%/40% by end of 2019 and 50/50 in the long term
– Aiming for better geographical/career-stage diversity
• Collecting data on career stage
• Explore blinding in the early stages of the process
• Explore if eLife review process is more or less biased
because of the consultation between reviewers
• Publish-Review-Curate
• Work with bioRxiv, ASAPbio, Center for Open Science
(RP:CB)
8. Meta-research papers in eLife
• Centralized scientific communities are less likely to generate
replicable results (Danchev et al. 2019. eLife 8:e43094)
• Gender variations in citation distributions in medicine are very small
and due to self-citation and journal prestige (Andersen et al. 2019.
eLife 8:e45374)
• Gender bias in scholarly peer review (Helmer et al. 2017. eLife
6:e21718)
• Gender inequalities among authors who contributed equally
(Broderick & Casadevall. 2019. eLife 8:e36399)
• The readability of scientific texts is decreasing over time (Plaven-
Sigray et al. 2017. eLife 6:e27725)
• Use of the Journal Impact Factor in academic review, promotion, and
tenure evaluations (McKiernan et al. 2019. eLife 8:e47338)
14. Outline
• Peer review at eLife
• Study 1: Author-reviewer homophily in peer review
• Study 2: Results from a peer-review trial
• What next?
• Meta-research papers in eLife
15. About eLife
â—Ź 700+ submissions per month
â—Ź 120+ publications per month
â—Ź Mike Eisen = EIC
â—Ź 60 Senior Editors
â—Ź 430+ Reviewing Editors
16. Editorial process
Full
submission
Peer review
Decision
after peer
review
Revision
assessed by
BRE
Assign to Reviewing Editor, who serves as a
reviewer and contacts other reviewers
Consultation amongst reviewers
Essential revisions listed
Limit rounds of revision
Editor's Notes
Funded by Wellcome (and HHMI, MPG, KAWF)
One could think of eLife as an experiments in meta-research - eLife was the intervention but there was no control!!
NO PAGE BUDGET
Acceptance rates: 15.6 for male corresponding authors; 13.8 for female corresponding authors
For both initial and full submissions, the prestige of the author's institution was the strongest predictor of a positive peer review outcome
Acceptance rates: 15.6 for male corresponding authors; 13.8 for female corresponding authors
For both initial and full submissions, the prestige of the author's institution was the strongest predictor of a positive peer review outcome
Outcomes were influenced by homophily — a preference of gatekeepers for manuscripts from authors with shared characteristics.
Experiment on consultation: does the first person to comment influence the outcome? Does the most senior person to comment influence the outcome?
Experiment on consultation: does the first person to comment influence the outcome? Does the most senior person to comment influence the outcome?
Danchev = Comparative Toxicogenomics Database and LINCSL1000 high-throughput experiments; 51,292 drug-gene interactions in 3363 articles
Andersen = 1.2 million papers in selected medical journals
Helmer = 9000 editors and 43000 reviewers from journals in the Frontiers series; women under-represented; homophily
Broderick = 2898 papers in selected
Plaven-Sigray = 700k abstracts in life and biomed journals
McKiernan = 864 documents from 129 universities
One could think of eLife as an experiments in meta-research - eLife was the intervention but there was no control!!