A pitch on directions to improve experimental reproducibility, illustrated by examples of past experiences. I made the plee to move from 'Proudly invented here' to 'Proudly copyied from', to re-use each other's eperiences in successes and failures.

1. Avoiding garbage in, garbage out
in multi-omics approaches towards impact
DTL partner event
“Future of data driven life sciences”
Utrecht, 23 November 2022
Prof. Alain van Gool
Professor Personalized Healthcare
Translational Metabolic Laboratory
Radboudumc Technology Centers
Netherlands X-omics Initiative
EATRIS Biomarker Platform

2. 2 DTL Partner Event, Utrecht, 23 nov 2023, Alain van Gool
A short story: A biomarker validation study
• Within Schering-Plough 4 Lead Optimisation programs in ERK pathway (2009)
• Need for biomarker that indicated downstream effects of drugs:
1. Inhibition ERK pathway (pharmacodynamic, mechanistic)
2. Tumor inhibition (efficacy, clinical)
3. Blood-based (circulating)
• Extensive transcriptomics profiling: IL-8 protein as promising candidate biomarker
Data for RAFi #4
RAFi
MEKi
ERKi
RAFi
#1
RAFi
#2
RAFi
#3
RAFi
#4
MEKi
#1
MEKi
#2
ERKi
#1

3. 3 DTL Partner Event, Utrecht, 23 nov 2023, Alain van Gool
Validation study to confirm IL-8 as biomarker in melanoma
Literature
{Yurkovetsky, et al. Clin Cancer Res, 2007}
Study objectives:
• Confirm elevated IL-8 protein in melanoma
• Develop IL-8 immunoassays for clinical use

4. 4 DTL Partner Event, Utrecht, 23 nov 2023, Alain van Gool
Validation study to confirm IL-8 as biomarker in melanoma
Stage 1 Stage 2 Stage 3 Stage 4
H&E staining; 20x
59 melanoma samples (tumor tissue (ffpe) + matching serum & plasma, stage I-IV,
from two independent biobanks) + 40 healthy serum & plasma samples
1. Genetic analysis for BRAFV600E/D mutation in genomic DNA from tissue
2. IL-8 mRNA analysis in tissue samples by in situ hybridisation using bDNA probes
(multiplexing with 12 ERK pathway response transcripts)
3. IL-8 protein analysis in tissue samples by immunohistochemistry (in parallel with
4 other ERK pathway response proteins, Ki67, Tunnel)
4. IL-8 protein analysis in matching plasma and serum by IL-8 immunoassay
(3 formats: ELISA, Luminex, Mesoscale; singleplex and multiplex; 2 standards)
OK
OK
?
OK (also WT BRAF?)

5. 5 DTL Partner Event, Utrecht, 23 nov 2023, Alain van Gool
Validation study to confirm IL-8 as biomarker in melanoma
Literature
{Yurkovetsky, et al. Clin Cancer Res, 2007}
Own data
{Unpublished, 2010}
Cause?
(6 months, 5 fte, USD 1.000.000)

6. Lessons learned?
Particularly for this case:
1. Know sample history
• IL-8 protein appeared sensitive to freeze-thawing
• History of our samples unknown?!
2. Know all relevant information from the source (patient)
• Tumor load may have been too low for our patients
• Tumor load of our donors unknown?!
3. We need to improve experimental reproducibility!
• Share burden, increase power, ensure better data
• Share lessons learned and do’s / don’t’s
• Avoid frustration and loss of time / money
{Stephan Nierkens, UMC Utrecht}
6 DTL Partner Event, Utrecht, 23 nov 2023, Alain van Gool

7. 7 DTL Partner Event, Utrecht, 23 nov 2023, Alain van Gool
Irreproducibility of data
{Freedman et al, PLOS Biology, 2015}
Half of published data cannot be reproduced!
Also see work from
Prof John Ioannidis (Stanford)

8. 8 DTL Partner Event, Utrecht, 23 nov 2023, Alain van Gool
Categories of errors leading to irreproducibility
{Freedman et al,
PLOS Biology, 2015}
From own experience:
Unawareness of critical steps in:
Pre-analytics:
• Sample generation
• Sample isolation
• Sample handling
• Sample storage
• Sample preparation
• Standards preparation
Analytics:
• Sample analysis
• Standards analysis
• Raw data processing
• Data stewardship at source
Post-analytics:
• Data integration and analysis
• Data interpretation
• Reporting

9. 9 DTL Partner Event, Utrecht, 23 nov 2023, Alain van Gool
But … crap data will remain crap data
even if made FAIR !
Big hopes for

10. 10 DTL Partner Event, Utrecht, 23 nov 2023, Alain van Gool
{van Gool et al, Nature Reviews Drug Discovery, Apr 2017}
Time for quality, not quantity !
10 DTL Partner Event, Utrecht, 23 nov 2023, Alain van Gool

11. The Netherlands X-omics Initiative
• National Roadmap Large Scale
Research Infrastructure
• 2018-2028
• €40M, partially funded by NWO
Objectives:
1. Advance X-omics technologies far beyond
state-of-the-art
2. Realize a genuinely integrated X-omics
infrastructure in NL
3. Develop and demonstrate impact of robust X-
omics analysis
Coordinator:
Alain van Gool

12. Example: population screening
▪ ERGO (Erasmus Rotterdam Gezondheid Onderzoek)
▪ 20.000 people 40+ years old in Rotterdam Ommoord
▪ Study: large scale test of 10.000 samples from ERGO biobank
▪ Glucose, cholesterol, hormones, electrolytes (Na, K)
▪ Analysis using clinical optimized protocols and auto-analysers
▪ Results: Na levels all a bit too high (?)
▪ Retest 2.000 samples: normal Na levels
▪ Probable reason after several test-retests:
▪ Electrolytes were usually measured last in the row of multiple tests (but not always), evaporation
yielded higher concentrations
▪ The order of measurement in the auto-analyzer affected the value obtained
▪ Consequence
▪ Frustration, loss of time and of 15.000 aliquots (some 20 years old and rare)
Prof Arfan Ikram
ErasmusMC

13. Example: Metabolomics of tissue of mouse models
• Question: Can we study tissue-specific pathophysiology in mice model
using metabolomics?
Example: sarcopenia and metabolic syndrome
• Observation: Variation between tissue samples from same mice ?!
• Explanation: Harvesting of organs of sacrificed mice is
laborious and time-consuming.
The order and time used for harvesting and processing
of organs affected the metabolomic profiles obtained.
• Consequence: frustration, loss of animals and time
• Measure: Check adenylate energy charge
He Y et al, Metabolites (2022) 12 (8): 742
Prof Thomas Hankemeier
Leiden University

14. X-omics community
• Information
• Exciting news
• New science
• Workshops, training
• Expert posts
• Events
• Etc
X-omics.nl
…

15. WP3:
Quality
Assessment
(Finland)
WP2:
Data Stewardship
& Integration
(The Netherlands)
WP1:
Multi-omic
technologies
(Checz Rep)
Multiomic toolbox: joining results and materials from WP1, 2 and 3
Flagship Project EATRIS-Plus:
creating new Omics resources
for Personalized Medicine

16. Multi-Omic Toolbox : Key Scientific Objective
Data Stewardship
& Integration (WP2)
Quality Assessment
(WP3)
Multi-omic
technologies (WP1)
▪ Genomic
▪ Epigenetic
▪ Metabolomic
▪ Proteomic
▪ Transcriptomic
Data
Generation
Data
Curation
Quality
Assessment
Data
Variability
Multi-Omic Toolbox
Academia & Industry
Pilot Access
• SOPs
• Guidelines for best practices
• Reference materials
• Quality parameters
• Data analytical tools
• Criteria for reference values
• Troubleshooting guidelines
• Repository of multi-omic data
An open access
resource, containing:
To enable high-quality research in the context of patient
stratification and accelerate PM solutions implementation.
Personalized Medicine
Oriented

17. Things to do still in omics biomarkers
• Further improve omics technologies (cell, individual, population)
• Increase analytical robustness of omics platforms
• Fully implement FAIR-data at source
• Increase capabilities to combine omics with other data
• Design and build improved (AI) models for complex data analysis
• Implement from research in health and healthcare (TRL 1 → 9)
• Share and use lessons learned
www.slideshare.net/alainvangool
17 DTL Partner Event, Utrecht, 23 nov 2023, Alain van Gool

18. 18 DTL Partner Event, Utrecht, 23 nov 2023, Alain van Gool
Different behaviour life scientists needed to improve quality
To discover or To confirm?

19. 19 DTL Partner Event, Utrecht, 23 nov 2023, Alain van Gool
Different behaviour life scientists needed to improve quality
“Every-one wants to innovate,
but no-one wants to change”
Bas Bloem (Radboudumc)
on clinical translation
“' We are really good at innovation
but bad in scaling”
Prins Constantijn van Oranje
at Health-RI conference 2021
“We should move from Proudly invented here
to Proudly copied from“
Alain van Gool
at DTL Partner Event 2022

20. Acknowledgements
Hans Wessels
Jolein Gloerich
Dirk Lefeber
Karlien Coene
Purva Kulkarni
Gadi Armony
Richard Rodenburg
Bert van den Heuvel
and others
Marcel Nelen
Albert Heck
Thomas Hankemeier
Peter Bram ‘t Hoen
Daniella Kasteel
Jessie Smits
and others
Collaborators/funders
Helger Ijntema
Alexander Hoischen
Lisenka Vissers
Christian Gillisen
Janneke Weiss
Han Brunner
alain.vangool@radboudumc.nl
www.radboudumc.nl/en/people/alain-van-gool
www.slideshare.net/alainvangool
Dapha Habets
Irene Keularts
Marek Noga
and others
Gary Kruppa
Pierre-Olivier Schmit
Dennis Trede
and others
Toni Andreu
Florence Bietrix
Emanuela Oldoni
Laura Garcia Bermejo
Andreas Scherer
Eliis Keidong
and others
Hans Jacobs
Peter-Bram ‘t Hoen
Anna Niehues
Xiaofeng Liao
Casper de Vries
Junda Huang
Translational Metabolic Laboratory
Human Genetics Nijmegen
Center for Molecular and Biomolecular Informatics