Reproducible Science - Panel at iEvoBio 2014

803 views

Published on

My introduction and position slides for the Reproducible Science panel at the 2014 iEvoBio conference, held June 24-25, 2014, in Raleigh, NC.

Published in: Science
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
803
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Reproducible Science - Panel at iEvoBio 2014

  1. 1. Reproducible Science Panel at iEvoBio 2014 Hilmar Lapp National Evolutionary Synthesis Center (NESCent)
  2. 2. Disclosure (or call it perspective) • Funded by NSF through BEACON for a series of workshops on developing reproducible science curriculum • 3 principle questions: • What practices, tools, and resources are available now? • How best to teach these? • What are the gaps faced by biologist users?
  3. 3. An experiment with sobering results https://storify.com/hlapp/reproducibility-repeatability-bigthink
  4. 4. An experiment with sobering results
  5. 5. Main takeaways (distilled to tweets) • software with many dependencies -> exponentially lower prob that all install • holes or errors in docs -> harmless for experts, often fatal for "method novice" • software evolution & rot -> parameters that worked 1 year ago now throw an error • Non-domain reproducers harder: baseline software, packages differ #dependencyhell
  6. 6. http://ropensci.org/blog/2014/06/09/ reproducibility/
  7. 7. “arguing that reproducibility is laudable in general glosses over the fact that for each research group it is a significant amount of work to make their research (easily) reproducible for independent scientists”
  8. 8. “Any work you do to make your analysis more reproducible pays dividends for colleagues and your future self.” Jeremy Leipzig
  9. 9. For research to be reproducible, the parts need to be available to start Collberg et al (2014), Measuring Reproducibility in Computer Systems Research. http://reproducibility.cs.arizona.edu/tr.pdf
  10. 10. A huge tech soup • vagrant • Ansible • Docker • Drone • Travis • knitr • packrat • VM memory limits • VM storage limits • VM uptime limits • firewalls • protected data • data snapshotting
  11. 11. Reproducible science is a huge opportunity for Research IT to enable & accelerate science.

×