This document discusses the development of a data management platform to improve the reproducibility of microscopy experiments. The platform aims to safekeep experimental data, allow sharing of data between research groups, and support reuse of data. It describes challenges to reproducibility and factors that influence it. The current prototype extends an existing open-source data management system to document experimental methods and parameters. Preliminary results suggest it facilitates organization and validation of scientific data. Future work includes adding more attributes, representing data in a machine-readable format, and integrating with workflow engines.

1. Towards Reproducibility of Microscopy Experiments
Sheeba Samuel, Frank Taubert, Daniel Walther, Birgitta K¨onig-Ries,
H. Martin B¨ucker
Institute for Computer Science
Friedrich-Schiller Universit¨at Jena
http://www.receptorlight.uni-jena.de/
RepScience, September 9, 2016
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 1

2. Motivation
An environment facilitating reproducibility for microscope experiments
for Collaborative Research Center ReceptorLight (CRC).
Aim: A data management platform for the CRC that
safekeeps the data produced in the individual subprojects
allows sharing of data among subprojects
supports data reuse by other CRC scientists
Conventional way to use (analog) lab notebooks in the scientific
community.
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 2

3. How good are we at performing experiments?
Image Source: https: // awritersden. files. wordpress. com
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 3

4. A normal experimental setup
Example: confocal patch clamp fluorometry (cPCF) experiment [1].
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 4

5. Repeatability and Reproducibility of Experiments
Repeatability: capability of getting the same results carried out by
the same experimenter using the same conditions of measurement. [2]
Reproducibility: capability of getting the same results carried out by
an independent experimenter using different conditions of
measurement.
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 5

6. Why Reproducibility is so important?
Studies show only 10% of published science articles are reproducible.
Scientists at the pharmaceutical company, Bayer, could reproduce
only 14 out of 67 projects. [3]
Studies conducted by the biotech company, Amgen, reveals that only
6 of the 53 studies were reproduced in Cancer Research. [4]
The US government gives nearly $31 billion every year in science
funding through NIH.
Image Source: http: // www. wakeupkiwi. com/
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 6

7. How to achieve reproducibility of experiments?
Data and methods used for the experiment
Workflow
Provenance
User annotations, labeling and tagging
Software used, data technologies and version control
Experiment environment parameters
Data sharing, archiving and distribution
Data storage
Machine-readable integration and documentation [5]
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 7

8. Related Work
Scientific workflow management system [6]
The work of Abramson et al. [7]
Pimentel’s work [8]
PROV-O [9]
OMERO [10]
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 8

9. Current Prototype and First Results
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 9

10. Current Prototype and First Results
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 10

11. Current Prototype and First Results
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 11

12. Current Prototype and First Results
Documentation of all necessary information regarding experiments
while performing them.
The data model consists of various input classes like
Experiment
Plasmid
Protein
Chemical Substance
Vector
Sharing and Improvement of this information by other scientists.
Extensible system - integration with less effort.
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 12

13. Current Prototype and First Results
The current prototype is developed based on OMERO.
Features of OMERO
Hierarchical organization of data
Projects, Datasets and images
Extended the OMERO server
An “Experiment“ tab to document and view all the information
regarding an experiment.
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 13

14. Current Prototype and First Results
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 14

15. Current Prototype and First Results
Developed a desktop client to deploy on workstations without
internet.
In the desktop client, a researcher while conducting an experiment
can input all the data.
upload the images, files and measurements obtained from the devices
during the experiment to the server when an internet connection is
available.
Minimizing the loss of data naturally occurring when recording these
things from memory.
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 15

16. Features of Current Prototype
Retreive and correlate all measurements obtained from different
devices belonging to one experiment.
View the images and data of the experimental protocol at one place
organized in a hierarchical manner.
Provides access control.
Different roles and permissions to restrict modification of data.
Sharing of data between users.
The version management of the scientific data.
Track modifications
Correction of mistakes
revert back to previous versions if needed
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 16

17. First Results
Manual validation by a group of biological scientists.
Being continuously extended and improved with the feedback received
from the scientists.
Image Source: http: // www. rwemarketing. com
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 17

18. Conclusion
Reproducibility of scientific experiments.
Ensure validation and correctness of biological scientist’s work.
A software platform supporting reproducibility.
Can spend more time on their biological work rather than focusing on
writing scripts and worrying about the storage of their huge data.
Specifically designed keeping in mind the requirements of biological
scientists working in the field of microscopy.
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 18

19. Future Work
Addition of more attributes to the current set based on the specific
requirements of further working groups of biological scientists.
To represent data entered by the scientists in a machine-readable
format.
Integration with existing tools to export the provenance information
from the system.
Integration with a workflow engine to represent the scientific workflow
of experiments.
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 19

20. Acknowledgement
This research is partially supported by the “Deutsche
Forschungsgemeinschaft“ (DFG) of the CRC
“High-end light microscopy elucidates membrane receptor function -
ReceptorLight“.
We thank our collaborators for providing the requirements to develop
the proposed approach and validating the system.
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 20

21. References I
Biskup, C., Kusch, J., Schulz, E., Nache, V., Schwede, F., Lehmann,
F., Hagen, V., Benndorf, K.: Relating ligand binding to activation
gating in cnga2 channels. Nature 446(7134), 440–443 (2007)
Taylor, B., Kuyatt, C.: Nist technical note 1279: Guidelines for
evaluating and expressing the uncertainty of nist measurements
results. National Institute of Standards and Technology, Washington
DC (1994)
Prinz, F., Schlange, T., Asadullah, K.: Believe it or not: how much
can we rely on published data on potential drug targets? Nature
reviews Drug discovery 10(9), 712–712 (2011)
Begley, C.G., Ellis, L.M.: Drug development: Raise standards for
preclinical cancer research. Nature 483(7391), 531–533 (2012)
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 21

22. References II
Goecks, J., Nekrutenko, A., Taylor, J.: Galaxy: a comprehensive
approach for supporting accessible, reproducible, and transparent
computational research in the life sciences. Genome biology 11(8), 1
(2010)
Curcin, V., Ghanem, M.: Scientific workflow systems-can one size fit
all? In: 2008 Cairo International Biomedical Engineering Conference,
pp. 1–9 (2008). IEEE
Abramson, D., Bethwaite, B., Dinh, M.N., Enticott, C., Firth, S.,
Garic, S., Harper, I., Lackmann, M., Nguyen, H., Ramdas, T., et al.:
Virtual microscopy and analysis using scientific workflows. In:
e-Science, 2009. e-Science’09. Fifth IEEE International Conference
On, pp. 239–246 (2009). IEEE
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 22

23. References III
Pimentel, J.F., Dey, S., McPhillips, T., Belhajjame, K., Koop, D.,
Murta, L., Braganholo, V., Lud¨ascher, B.: Yin & yang: demonstrating
complementary provenance from noworkflow & yesworkflow. In:
International Provenance and Annotation Workshop, pp. 161–165
(2016). Springer
Lebo, T., Sahoo, S., McGuinness, D., Belhajjame, K., Cheney, J.,
Corsar, D., Garijo, D., Soiland-Reyes, S., Zednik, S., Zhao, J.: Prov-o:
The prov ontology. W3C Recommendation 30 (2013)
Allan, C., Burel, J.-M., Moore, J., Blackburn, C., Linkert, M.,
Loynton, S., MacDonald, D., Moore, W.J., Neves, C., Patterson, A.,
et al.: Omero: flexible, model-driven data management for
experimental biology. Nature methods 9(3), 245–253 (2012)
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 23

24. Thanks
Towards Reproducibility of Microscopy Experiments RepScience, September 9, 2016 24