Your SlideShare is downloading. ×
Berlin 6 Open Access Conference: Jelena Kovacevic
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Berlin 6 Open Access Conference: Jelena Kovacevic

423
views

Published on

www.berlin6.org

www.berlin6.org

Published in: Education, Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
423
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Jelena Kovačević: MR Tools for Next-Generation Bioinformatics Systems 06/06/09 This is a brief version of the presentation I gave at Icassp 2007 in the special session on reproducible research. My goal was to give some input based on my experience during the 5 years I served as the Editor-in-Chief of the IEEE Transactions on Image Processing.
  • Transcript

    • 1. How to Encourage and Publish Reproducible Research Jelena Kovačević Center for Bioimage Informatics Department of Biomedical Engineering Department of Electrical & Computer Engineering Carnegie Mellon University
    • 2. Theory Vs Experimentation
      • Theoretical disciplines
        • Mathematics
      • Experimental disciplines
        • Biology
      • A hybrid is born: Computational sciences
        • Should follow good practices from both
        • SP falls in there: How are we doing?
      Axioms Lemmas Proof MIT CMU Stanford Harvard hypothesis
    • 3. Issues
      • Cultural
        • Innovation above all else
        • TIP Transactions reviewing questions
          • 1. Is the paper technically sound?
          • 2. Is the coverage of the topic sufficiently comprehensive and balanced?
          • 3. How would you describe the technical depth of the paper?
          • 4. How would you rate the technical novelty of the paper?
        • Can lead to paradox
      • Educational
        • Our students undertrained in statistics
        • Typically reimplement everything
      • Data
        • We collaborate and data might not be ours
      • IP
        • Data issues
        • Companies and agencies protecting their IP
      • Collaborative
        • With colleagues within the university/company, outside
    • 4. How Do We Publish RR?
      • Not likely to happen overnight
        • Encourage and reward “good behavior” (Child psychology 101)
      • Ideas
        • Special section in Transactions for RR?
        • Establish a paper award for an RR paper?
        • Form a rough guideline of what each paper should contain for an RR designation?
        • Everything we read is partly “on faith”
    • 5. How to Make Papers RR?
        • Used in my group
        • Compilation of ideas from Barni and EPFL groups (Vetterli, Vandewalle et al.)
        • Compendium (Gentleman & Lang)
        • Freeze the code upon
          • Submission
          • Acceptance
        • “ Good intentions” enforced
        • Students do projects and reproduce
    • 6. An Entirely NonRR Case Study
      • Data set
        • 15 papers published in the TIP
        • EDICS category using both theory and experimentation
        • Stayed away from standards as well as biomed
        • For all algorithms, competing ones exist
        • RR
          • block-diagram?
          • pseudo code?
          • data available?
          • code available?
          • proof available?
      • Ratings (0, 0.5, 1)
        • Algorithm and experimental setup
          • algorithm explained?
          • data explained?
          • data size?
          • details on parameters used?
          • comparison to competing algorithms?
    • 7. Results of the Entirely NonRR Case Study
        • All papers had proofs, none had code available
        • Sufficient detail on algorithms, none had a block-diagram
        • Data used, data size and availability all below average
        • Half of the cases were the parameters specified
        • Comparisons to competing algorithms: quarter
        • Pleasant surprise: 60%, pseudo-code was available