Your SlideShare is downloading. ×
Berlin 6 Open Access Conference: Jelena Kovacevic
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Berlin 6 Open Access Conference: Jelena Kovacevic


Published on

Published in: Education, Technology

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • Jelena Kovačević: MR Tools for Next-Generation Bioinformatics Systems 06/06/09 This is a brief version of the presentation I gave at Icassp 2007 in the special session on reproducible research. My goal was to give some input based on my experience during the 5 years I served as the Editor-in-Chief of the IEEE Transactions on Image Processing.
  • Transcript

    • 1. How to Encourage and Publish Reproducible Research Jelena Kovačević Center for Bioimage Informatics Department of Biomedical Engineering Department of Electrical & Computer Engineering Carnegie Mellon University
    • 2. Theory Vs Experimentation
      • Theoretical disciplines
        • Mathematics
      • Experimental disciplines
        • Biology
      • A hybrid is born: Computational sciences
        • Should follow good practices from both
        • SP falls in there: How are we doing?
      Axioms Lemmas Proof MIT CMU Stanford Harvard hypothesis
    • 3. Issues
      • Cultural
        • Innovation above all else
        • TIP Transactions reviewing questions
          • 1. Is the paper technically sound?
          • 2. Is the coverage of the topic sufficiently comprehensive and balanced?
          • 3. How would you describe the technical depth of the paper?
          • 4. How would you rate the technical novelty of the paper?
        • Can lead to paradox
      • Educational
        • Our students undertrained in statistics
        • Typically reimplement everything
      • Data
        • We collaborate and data might not be ours
      • IP
        • Data issues
        • Companies and agencies protecting their IP
      • Collaborative
        • With colleagues within the university/company, outside
    • 4. How Do We Publish RR?
      • Not likely to happen overnight
        • Encourage and reward “good behavior” (Child psychology 101)
      • Ideas
        • Special section in Transactions for RR?
        • Establish a paper award for an RR paper?
        • Form a rough guideline of what each paper should contain for an RR designation?
        • Everything we read is partly “on faith”
    • 5. How to Make Papers RR?
        • Used in my group
        • Compilation of ideas from Barni and EPFL groups (Vetterli, Vandewalle et al.)
        • Compendium (Gentleman & Lang)
        • Freeze the code upon
          • Submission
          • Acceptance
        • “ Good intentions” enforced
        • Students do projects and reproduce
    • 6. An Entirely NonRR Case Study
      • Data set
        • 15 papers published in the TIP
        • EDICS category using both theory and experimentation
        • Stayed away from standards as well as biomed
        • For all algorithms, competing ones exist
        • RR
          • block-diagram?
          • pseudo code?
          • data available?
          • code available?
          • proof available?
      • Ratings (0, 0.5, 1)
        • Algorithm and experimental setup
          • algorithm explained?
          • data explained?
          • data size?
          • details on parameters used?
          • comparison to competing algorithms?
    • 7. Results of the Entirely NonRR Case Study
        • All papers had proofs, none had code available
        • Sufficient detail on algorithms, none had a block-diagram
        • Data used, data size and availability all below average
        • Half of the cases were the parameters specified
        • Comparisons to competing algorithms: quarter
        • Pleasant surprise: 60%, pseudo-code was available