BOSC 2012 panel discussion
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

BOSC 2012 panel discussion

on

  • 4,103 views

Should we have standard bioinformatics review criteria?

Should we have standard bioinformatics review criteria?

Statistics

Views

Total Views
4,103
Views on SlideShare
4,082
Embed Views
21

Actions

Likes
0
Downloads
5
Comments
0

5 Embeds 21

https://twitter.com 16
https://twimg0-a.akamaihd.net 2
https://si0.twimg.com 1
http://www.twylah.com 1
http://dev.plu.mx 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

BOSC 2012 panel discussion Presentation Transcript

  • 1. Some anecdotes Wanted to build on a very notorious bioinformatics paper; asked for pipeline script; main author sent us a Perl script that had a syntax error in the third line. (<- joke goes here) Was talking with collaborators about digital normalization, they said “oh, we’ve been doing that for ages. Here are refs.” Refs were to high profile papers that said “we filtered k-mers.” No software, no parameters, no validation. High profile paper on metagenome assembly that used computational spike-in data (concluding that existing approaches had low sensitivity). Asked for data; no record of it.A very real problem is that there is no recourse in these situations. There is no way to check if these people are doing good science or not. WTH???
  • 2. Focus on replication issue Most journals already require data, methods (software) to be available to reviewers and readers.  http://ivory.idyll.org/blog/journal-data-policies.html (#bosc12)  Although for software, this is most often honored in the breach… It is easy to justify this for replication and good science! Note, no explicit requirement to allow modification & distribution. Bigger goals:  A gateway path to open source, reusability?  Developers are nervous about releasing crappy code, so code quality will almost certainly improve.
  • 3. So: Should there be review standards for bioinformatics? YES. What should they focus on? REPLICATION DURING REVIEW. For what kinds of bioinformatics? ANY ARTICLE WHERE UNPUBLISHED SOFTWARE IS USED. How should they work? REVIEWER OPT-IN.  No journal infrastructure required.  Allows for judgement of reviewer to play a role.  Most editors are timid and will not overrule reviewer, especially when their own journal guidelines support.  If you can get 1 reviewer /paper to do it, you have changed the field.
  • 4. A specific proposal A position statement and/or white paper from OBF and others. A JS/static Web site (no DB/tracking!) with  Checklist for reviewers that generates score and back- link;  Links to position statement & journal policies;  Dynamically generate summary paragraph with score, that links back to Web page. Encourage reviewers (us!!) to use as opt-in component of review.  Can be one component, minor or major, of review, at reviewer discretion;  Serves as polite education rather than merely
  • 5. An example paragraph.Bioinformatics review criterion: The software is available, +1; the software license allows readers to run it, +1; the software is available, + 1; I didnt download it. The data is available, +1; the data format is straightforward, +1; I didnt download the data.Score: 5/5. See http://short/url/to/brc for more information.