The future of Peer Review: ensuring/assuring quality thoughts of a “freshman” editor


Published on

Revisão pelos pares é uma “bênção”1 sobre a ciência que tem sido revisada pelo seu mérito para publicação. “A fase de revisão por pares continuará a ser essencial para garantir que o corpo da ciência cresça através de descobertas e afirmações reais e embasadas. A revisão por pares exclui desinformação prejudicial enquanto se adapta a novos resultados… em última análise, é um serviço extraordinário provido gratuitamente pelos cientistas para a comunidade científica e a sociedade como um todo”.2 Entretanto, seu processo é inerentemente frágil, sendo dependente de um revisor selecionado por seu/ sua experiência dentro do campo, mas que não é remunerado para a realização da tarefa, deve encontrar tempo para a tarefa em meio a uma carreira/vida pessoal movimentada, e pode ter, até certo ponto, perdido perícia (com o aumento da senioridade sua/seu engajamento e atividade em pesquisa pode diminuir).

Peer review is a “blessing”1 upon the science that has been reviewed for its worthiness for publication. “The peer-review stage will continue to be essential for ensuring that the body of science grows through real, supported discoveries and assertions. Peer review excludes damaging misinformation while adapting to new inputs … ultimately it is an extraordinary service provided for free by scientists to the scientific community and society as a whole.”2

Yet, the process is inherently fragile, being reliant on a reviewer selected for his/her expertise within the field but who is unpaid for undertaking the task, must carve time for the task out of a busy career/personal life, and may have, to some extent, lost expertise (with increasing seniority his/her engagement and activity in research may loosen).

La revisión por pares es una “bendición”1 sobre la ciencia que ha sido revisada por su mérito para su publicación. “El escenario de revisión por pares seguirá siendo esencial para asegurar que el cuerpo de la ciencia crezca a través de descubrimientos y afirmaciones reales y respaldadas. La revisión por pares excluye la falsa información perjudicial, mientras que adaptándose a los nuevos aportes … en última instancia, es un servicio extraordinario proporcionado gratuitamente por los científicos a la comunidad científica y la sociedad en su conjunto”.2

Sin embargo, el proceso es de por sí frágil, siendo dependiente de un revisor seleccionado por su experiencia en el campo, pero que no se le paga para llevar a cabo la tarea, debe extraer el tiempo para la tarea de una carrera o vida personal ocupada, y puede haber perdido, en cierta medida, conocimientos (con el aumento de la edad su compromiso y actividad en la investigación puede aflojarse).

Published in: Technology, Health & Medicine
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The future of Peer Review: ensuring/assuring quality thoughts of a “freshman” editor

  1. 1. The future of Peer Review ensuring/assuring quality thoughts of a “freshman” editor Janet Seggie, HMPG (SAMA)
  2. 2. South African Medical Journal South African Journal of Surgery South African Journal of Child Health South African Journal of Obstetrics and Gynaecology Southern African Journal of HIV Medicine South African Journal of Sports Science South African Journal of Bioethics and Law online only South African Journal of Psychiatry African Journal of Health Professions Education online only South African Journal of Radiology South African Journal of Communication Disorders Southern African Journal of Critical Care Continuing Medical Education not peer reviewed.
  3. 3. The future … - Is there a future for peer-review? - What will be the shape of peerreview in the future? the wisdom of the crowds vs wisdom of the few Wisdom of the crowds Nature (2006) | doi:10.1038/nature04992
  4. 4. “Game of Papers” “Cascading” peer review systems The consortia will enable papers, with their accompanying referee reports, to move more easily between publishers.
  5. 5. Who are the peer reviewers ? An “elite” - clinicians/scientists who work in a field relevant to the paper under consideration academics (on a relatively short list of experts/often in a narrow field) who have experience, insight, and the respect of other researchers – offer wise advice and get it right most of the time
  6. 6. The traditional process: wisdom of the few reward reward The Lancet Author submits Re-submits “in house” p r/triage certification & registration Acceptance (5%); rejection (75%); sent for P-R Re-review 6 expert reviewers (!) + statistician PUBLISH Call for revision(s) awareness archiving
  7. 7. Pre-publication peer review - clarifies - scientific error filtered out - poor quality work rejected - places new work in context of available evidence - requires declaration of limitations - prevents over-interpretation - It is right for the readers  journal fulfills the awareness role  published work cited & archived
  8. 8. a “flawed process at the heart of science and journals” • • • • Richard Smith Slow Expensive … of time and money Inefficient … cycling to other jnls Inconsistent (6 reviews for stats consistency!) • Subjective • Inexpert/amateurish • Blocks innovation • Biased … gender, language, nationality, against `negative studies’ • Open to abuse … conflicts of interest; politics • Fails to determine fraud
  9. 9. process is inherently fragile reliant on a reviewer selected for his/her expertise but who … - is unpaid for undertaking the task - receives no academic recognition/reward - must carve 4-8 hours for the task out of a busy career/ personal life - may have lost expertise … with increasing seniority his/her engagement & activity in research may loosen
  10. 10. The future …
  11. 11. the shape of peer-review in the future? 1  research/new findings in context … reports of new research linked to an up-to-date systematic review - the Cochrane collaboration model (at the Cochrane Library) … pre- and postpublication review - commissioned editorial
  12. 12. the shape of peer-review in the future? 2 • Open peer review = transparency where the identity of the reviewer (s) is known to the author … allows authors and readers to determine whether the review process has been just ! Smaller jnls c smaller constituencies … capacity ?
  13. 13. The shape of peer-review in the future 3 Replace the wisdom of the few with the wisdom of the crowds define 'peer' as broadly as possible – to maximize the power of collective intelligence The Internet – digital,networked environment
  14. 14. the “online peer” taps into something that already exists: journal clubs
  15. 15. pre-publication peer review “Disruption!” pre-publication only verify whether experiments and data analysis were conducted rigorously If published, papers made available for community-based open peer review … the scientific community ascertains importance through debate and comment. PLOS ONE publishes approximately 70 % of all submissions, after review by 2.8 experts.
  16. 16. encourages open peer-review Criteria for Publication reviewers seek: Originality Importance to researchers or practitioners in the field Interest for researchers or practitioners outside the field Rigorous methodology with substantial evidence for conclusions Conducted according to the highest ethical standards
  17. 17. “Game of Papers” “Cascading” peer review systems The consortia will enable papers, with their accompanying referee reports, to move more easily between publishers.
  18. 18. pre-publication peer review … a few, brave journal editors have suggested that peer review … decides where a study gets published rather than whether it gets published
  19. 19. Res Altwegg Dieter Oschadleus Les Underhill
  20. 20. Authors submit a manuscript Peers voluntarily engage in review processes Peerage Essay Peer-review-of-peer-review each review gets a quality index Authors and the Editors track the process – can access the review Manuscript revision upload, or withdrawal for re-submission Final evaluation of the revised manuscript Authors may accept a direct publishing offer from a subscribing journal or choose to export the peer reviews to any journal of their choice 6 weeks
  21. 21. pre-publication peer review a bias to publish BMC Medicine operates an 'open peer review' policy meaning reviewers are asked to sign their reviews. The pre-publication history including all submitted versions, reviewers' reports and authors' responses will be linked to form the published article. Open access Research articles published in BioMed Central's journals are freely available online to the entire global research community: - BioMed Central's website - the National Institutes of Health's electronic depository - PubMed Central
  22. 22. Post-publication via the Internet …
  23. 23. Post-publication on the Internet … a virtual journal club – correction of error in publications “we show you when there are new papers posted on Pubmed related to your interests and when there are new discussions about those papers”
  24. 24. Grad students and post-docs … contribute by sharing their insights about published data that regularly occur in lab meetings and journal clubs by summarizing papers and leaving comments build name recognition
  25. 25. “Being a part of a virtual journal club is pretty awesome. I get to focus on an area that I'm interested in with others, stay up on the latest research, and get a diverse array of expert opinions. And it's with scientists all over the world. This is the way reviewing science should be.” ~ Matt, Cell Biologist
  26. 26. Peer review via the internet … • • • • • • • • Faster Expensive … of time and money more efficient Inconsistent Subjective Inexpert/amateurish Blocks innovation Biased … gender, language, nationality, against `negative studies’ • Open to abuse … conflicts of interest; politics • Fails to determine fraud
  27. 27. The future … pre- and post-publication peer review “making continuous but constructive criticism of research a new norm of science” (Horton, 2011 – written evidence to UK Parliament) “blessing” upon the science that has been reviewed for its worthiness for publication Richard Smith
  28. 28. The shape of peer-review in the future aimed at improvement • • • • • • • • • • • standardizing procedures; opening up the process; blinding reviewers to the identity of authors; reviewing protocols; formal training of reviewers – standards and ethics being more rigorous in selecting and deselecting reviewers; using electronic review; rewarding reviewers; providing detailed feedback to reviewers; using more checklists; creating professional review agencies … capacity ?