Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

In review, unlocking the potential of early sharing


Published on

Presentation on open peer review, presented by Damain Pattinson at the Oxford OpenCon satellite event, November 2018.

  • Be the first to comment

  • Be the first to like this

In review, unlocking the potential of early sharing

  1. 1. 0 IN REVIEW – UNLOCKING THE POTENTIAL OF EARLY SHARING Damian Pattinson & Amye Kennall OpenCon Oxford November 2018
  2. 2. 1 0 5000 10000 15000 20000 25000 30000 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 TotalPublications Preprints Data from Preprints, yay!
  3. 3. 2 0 500000 1000000 1500000 2000000 2500000 3000000 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 TotalPublications Axis Title Papers Preprints Preprints, meh
  4. 4. 3 Even after a Statement on data sharing during public health emergencies signed by Springer Nature, Wellcome, and many others, only 3% of publications across both outbreaks were associated with preprints. Of those that were made available, most were available more than 100 days before publication. Publications with preprints — the Ebola and Zika outbreaks
  5. 5. 4 More than a nice practice, sharing early is a necessary condition of openness
  6. 6. 5 Springer Nature / Research Square history 2000 20172000 2002 2004 2006 2008 2010 2012 2014 2016 Nature Precedings Protocol Exchange Under Consideration from Nature Comms BMC experiments with early preprint via "Deposited Article" Partnership with bioRxiv American Journal Experts Rubriq Journal Guide Journal checks
  7. 7. 6 Powered by Research Square and developed in partnership with BMC, In Review aims to open up the peer review process to authors and beyond. The first service of its kind, In Review will provide authors with on-demand access to the granular status of their manuscript, including number of reviewers invited and immediate access to reviewer reports, and will allow authors to showcase their work to funders and others and to engage the wider community for comment and collaboration while their manuscript is under review. The In Review platform
  8. 8. 7 Author opt-in
  9. 9. 8 Public Article page Editorial badges Journal status annotations Review history – updated in real time whenever there is a status change
  10. 10. 9 Author view Suggest Reviewers: Allows authors to suggest reviewers if their paper is held up Full Peer Review History: Updated as soon as a review or decision is received Reader comments incorporated into the review package sent to editor
  11. 11. 10 Journal In Review browse page
  12. 12. 11 Feedback so far • First opt in 2 hrs after launching • Over 100 articles opted in within first 3 weeks • Opt in rate to early sharing: 3% à 46% • Very positive author feedback Journal Submissions to "In Review" Total Submissions (since 18 Oct) Opt-in rate (%) Trials 39 169 23% BMC Neurology 61 121 50% BMC Anaesthesiology 60 96 63% BMC Ophthalmology 68 115 59% TOTAL 228 501 46% Pilot opt in rate, 29 November
  13. 13. 12 Majority of opt-ins are coming from China and Asia generally 53% 12% 2% 4% 16% 13% China Europe United Kingdom United States Asia (exc China) Other #TrialsJournal launches innovative #preprint #platform. Great idea! @MedicalEvidence @shauntreweek @GrimshawJeremy Tianjing Li @JohnsHopkinsEPI Very interesting concept. Hope it becomes the new standard so we can stop frantically checking our papers' status. #open #PeerReview This is a great pilot! Your work gets a DOI (similar to a preprint) while it is undergoing peer review @BioMedCentral “Cool. It’s a little like F1000’s approach but on steroids.” Positive qualitative feedback
  14. 14. 13 Next steps • Expand journal participation—at Springer Nature and beyond—creating a cross- publisher service • Migration of Protocols Exchange and Nature Precedings (early January) • Offer ‘Direct Submission’ pathway so that authors can post preprints directly • Offer author tools such as automated language assessment and reproducibility checks • Expand community review functionality
  15. 15. 14
  16. 16. 15
  17. 17. 16 Structured review form RESEARCH SQUARE INDEPENDENT VALIDATION | Sample 1 RESEARCH QUALITY EVALUATION FAIL MANUSCRIPT TITLE redacted (keywords: QOL, mothers of children with disabilities) AUTHORS redacted REPORT DATE Mar 27 2018, 07:06 (UTC -0400) EVALUATED BY Academic Peer Reviewer Current Manuscript Version OBJECTIVE Does the manuscript have a clear objective? Yes - there is a clear objective DESIGN Is the current approach (including controls / protocols) appropriate for the objective(s)? No - there are major issues EXECUTION Are the experiments and analyses performed with technical rigor to allow confidence in the results? No - there are major issues INTERPRETATION Is the current interpretation of the results reasonable and not overstated? No - there are major issues Manuscript Potential Could an appropriately REVISED version of this work represent a technically sound contribution? No - manuscript has some fundamental flaw(s) REVIEWER COMMENTS This is a study comparing psychiatric diagnoses, quality of life, and global assessment of functioning of mothers of children in 3 groups: children with ASD without intellectual disability, children with ID without ASD and typically developing. This is clearly an important question, however, the study has significant limitations and the manuscript needs to be a revised by a person who is familiar with both the English language and with writing scientific papers. What have the authors done well: The authors identified an important question, and made an attempt to distinguish between mother of children with ASD without ID, and mother of children with ID. Unfortunately, this study has significant limitations, and the manuscript itself is poorly written. Issues with the study itself: 1. What is the rationale for this study, at one point the authors state that this question has not been investigated in the Arab world, but then they cite studies which were in fact done in the Arab world, but state that there were methodological problems. 2. The sample sizes are quite small for this kind of study, especially if this is not the first study of its kind. Effect sizes are not reported. 3. The authors state that the children with ASD had no intellectual disability, but did they have any other co-morbidities? It is stated that the ID group did not have any medical or psychiatric comorbidities, but this information is not provided for the ASD and typically developing groups. 4. The control group was recruited from social contacts of the ASD and ID groups - this could cause bias, since social contacts can compare their lives to the lives of their friends. A better control group would have been selected from the general population. 5. There was a significant difference in age between the children in the 3 groups - this could significantly affect results. 6. It is not mentioned whether the children had siblings, what were the ages of the siblings and whether the siblings had any diagnoses of their own. This information would significantly affect the well-being of the mothers.
  18. 18. 17 Peer review channels Unstructured Comments Hypothesis Annotation Structured Reviews Invited Reviews - Allows readers to ask informal questions to the authors. - Threaded to allow replies from authors and other readers. - Unmoderated but with ability to report unsuitable content. - Allows readers to make in-line annotations. - Comments can be collated and delivered to editor. - Standard moderation channels. - Free to anyone to submit. - Reviewers must complete a full RQE using our structured template. - Reviews are verified before posting. - Invited by the assigned journal editor. - Can take any form: RQE, unstructured or other. - Includes editor decision letters.
  19. 19. 18
  20. 20. 19