Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

On the Reproducibility of Science


Published on

Melissa Haendel
March 18, 2013
Beyond the PDF2 - Amsterdam, NL

Science, Publishing, Open Access

Published in: Technology
  • Be the first to comment

  • Be the first to like this

On the Reproducibility of Science

  1. 1. On the reproducibilityof scienceMelissa HaendelBeyond the PDF220 March
  2. 2. Do we know if the infrastructure isactually broken?Slide  from  Gully  Burns  The  science  cycle  
  3. 3. This is a broken data story.The  science  cycle  Image:  h6p://­‐on-­‐the-­‐pathway-­‐to-­‐ci=zenship  
  4. 4. Journal guidelines for methods areoften poor and space is limited“All  companies  from  which  materials  were  obtained  should  be  listed.”   -­‐  A  well-­‐known  journal  Reproducibility  is  dependent  at  a  minimum,  on  using  the  same  resources.  But…  
  5. 5. Hypothesis:  AnAbodies  in  the  published  literature  are  not  uniquely  idenAfiable    An experiment in reproducibilityGather  journal  ar=cles  5  domains:  Immunology  Cell  biology  Neuroscience  Developmental  biology  General  biology  3  impact  factors:  High  Medium  Low  28  Journals  119  papers  454  an=bodies  408  commercial  an=bodies  46  non-­‐commercial  an=bodies  Iden=fying  ques=ons:  Is  the  an=body  iden=fiable  in  the  vendor  site?  Is  the  catalog  number  reported?  Is  the  source  organism  reported?  Is  the  an=body  target  iden=fiable?  
  6. 6. The data shows…Approximately  half  of  anAbodies  are  not  uniquely  idenAfiable  in  119  publicaAons  Percent  idenAfiable  0%  10%  20%  30%  40%  50%  60%  Commercial  an=body   Non-­‐commerical  an=body  n=408  n=46  
  7. 7. 0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100%  Immunology  Neuroscience   Dev  Bio   Cell  Bio   General  Bio  High  Medium  Low  Percent  iden=fiable  n=124   n=94  n=87  n=95  n=56  Unique  idenAficaAon  of  commercial  anAbodies  varies  across  discipline  and  impact  factor  In some domains high impact journals have worsereporting, and in others it is the opposite
  8. 8. Maybe labs are just disorganized?
  9. 9. Meet the Urban Lab
  10. 10. Meet the Urban LabImage:  Gourami  Watcher  
  11. 11. A+ organization!The  Urban  lab  anAbodies  
  12. 12. Of 14 antibodies published in 45 articles,only 38% were identifiable 0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  Commerical  Ab  iden=fiable  Non-­‐commercial  Ab  iden=fiable    Catalog  number  reported  Source  organism  reported  Target  uniquely  iden=fiable  Percent  idenAfiable  
  13. 13. What does this tell us?
  14. 14. Scientists really do put theirdata in cardboard boxes.
  15. 15. Ø Promote  beJer  reporAng  guidelines  in  journals  Ø Include  reviewing  guidelines  Ø Provide  tools  to  reference  research  resources  with  unique  and  persistent  IDs/URIs    Ø Train  librarians  and  other  data  stewards  to  apply  data  standards  What are we going to do about it?