Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Dealing With Noise in Science


Published on

With increasing number of science journals and articles how can we build a post-publication evaluation system ?

Published in: Economy & Finance, Technology
  • Be the first to comment

Dealing With Noise in Science

  1. 1. Dealing with Noise How to build a post-publication evaluation system
  2. 2. PDF downloads Nature Genetics ( N = 168) for 2005. R = 0.71 From: Turning web traffic into citations Action Potential blog
  3. 3. Connotea tag:Evolution November 2006 - Papers in Connotea with tag "evolution” published 2003 or 2004 Early 2008 - Papers in Connotea with tag "evolution” published 2005 or 2006 Journal Articles Citations MBE (2003/2004) 480 6771 Nature (2003/2004) 5193 121932 Science (2003/2004) 5306 124377 Connotea Evolution (2003-2004) 86 3243 Connotea Evolution (2005-2006) 149 5059
  4. 4. Bloggers talk about interesting papers There are 519 papers published in 170 journals in the period of 2003-2004 that were mentioned in blogs tracked by Postgenomic.
  5. 5. Quotes “ (…)what is obvious to everyone but bone-headed bean counters. The only way to assess the merit of a paper is to ask a selection of experts in the field. “ David Colquhoun Improbable Science “ In order to shortcut the work of looking up actual (real) citation counts for investigators the journal impact factor is used as a surrogate to estimate the count. I have always warned against this use”. Eugene Garfield (1998) “ Any measurement system will produce gamesmanship to increase success. Lawrence yearns for the one system that obviates it: face-to-face conversations between peers that allows a truly deep and well focused estimate of scientific merit. “ Philip Campbell
  6. 6. Dealing with Noise Peer-review Nature, Science, Cell Lab notebooks, unpublished manuscripts Journal impact factor Perceived impact
  7. 7. Dealing with Noise Perceived impact Peer-review Highly recommended papers by your peers Online lab notebooks Pre-print servers Aggregated usage data
  8. 8. Post-publication journals (…) nobody wants to have to wade through a morass of papers of hugely mixed quality, so how will the more interesting papers in such an archive get noticed as such? Philip Campbell
  9. 9. Post-publication journals Automatic Algorithms Editors Hubs/Portals