Dealing with Noise How to build a post-publication evaluation system
PDF downloads Nature Genetics ( N = 168) for 2005. R = 0.71 From: Turning web traffic into citations Action Potential blog
Connotea tag:Evolution November 2006 - Papers in Connotea with tag "evolution” published 2003 or 2004 Early 2008 - Papers in Connotea with tag "evolution” published 2005 or 2006 Journal Articles Citations MBE (2003/2004) 480 6771 Nature (2003/2004) 5193 121932 Science (2003/2004) 5306 124377 Connotea Evolution (2003-2004) 86 3243 Connotea Evolution (2005-2006) 149 5059
Bloggers talk about interesting papers There are 519 papers published in 170 journals in the period of 2003-2004 that were mentioned in blogs tracked by Postgenomic.
Quotes “ (…)what is obvious to everyone but bone-headed bean counters. The only way to assess the merit of a paper is to ask a selection of experts in the field. “ David Colquhoun Improbable Science “ In order to shortcut the work of looking up actual (real) citation counts for investigators the journal impact factor is used as a surrogate to estimate the count. I have always warned against this use”. Eugene Garfield (1998) “ Any measurement system will produce gamesmanship to increase success. Lawrence yearns for the one system that obviates it: face-to-face conversations between peers that allows a truly deep and well focused estimate of scientific merit. “ Philip Campbell
Dealing with Noise Perceived impact Peer-review Highly recommended papers by your peers Online lab notebooks Pre-print servers Aggregated usage data
Post-publication journals (…) nobody wants to have to wade through a morass of papers of hugely mixed quality, so how will the more interesting papers in such an archive get noticed as such? Philip Campbell