SlideShare a Scribd company logo
Preserving Software
Challenges and Opportunities
for Reproducibility of Science and Technology
Roberto Di Cosmo
Scilabtec 2015 - Paris
Irill - INRIA - University Paris Diderot
roberto@dicosmo.org
21st of May 2015
Roberto Di Cosmo Preserving Software for Reproductibility 1/26
How we built our scientific knowledge
The experimental method
make an observation
formulate an hypothesis
set up an experiment
formulate a theory
And then we reproduce and verify.
Reproducibility is the key
non-reproducible single occurrences are of
no significance to science
Karl Popper, The Logic of Scientific Discovery, 1934
Roberto Di Cosmo Preserving Software for Reproductibility 3/26
Reproducibility, today
Reproducibility (Wikipedia)
the ability of an entire experiment or study to be reproduced,
either by the researcher or by someone else working independently.
It is one of the main principles of the scientific method.
Why we want it
foundation of the scientific method
accelerator of research: allows to build upon previous work
visibility reproducible results are cited more often
Piwowar et al. Sharing Detailed Research Data Is Associated with
Increased Citation Rate. PLoS ONE 2(3): e308, 2007
transparency of results eases acceptance
necessary for industrial transfer
reproducibility is the essence of industry!
Roberto Di Cosmo Preserving Software for Reproductibility 4/26
Today: Software is Science’s cornerstone
Software is an essential component of modern scientific research
Deep knowledge from all fields is embodied in complex software
systems
Top 100 papers (Nature, October 2014)
[...] the vast majority describe experimen-
tal methods or software that have become
essential in their fields.
http: // www. nature. com/ news/
the-top-100-papers-1. 16224
Software is the pillar of science
Roberto Di Cosmo Preserving Software for Reproductibility 6/26
Software is Pervasive
At the heart of technology
televisions/fridges ≈ 10M SLOC
phones ≈ 20M SLOC
cars ≈ 100M SLOC
IoT ...
Key mediator for accessing all information (c) Banski
Information is a main pillar of our modern societies.
Absent an ability to correctly interpret dig-
ital information, we are left with [...] “rot-
ting bits” [...] of no value.
Vinton G. Cerf IEEE 2011
Software is an essential enabler for all technology
Roberto Di Cosmo Preserving Software for Reproductibility 7/26
Reproducibility in the digital age
For an experiment involving software, we need
open access to the scientific article describing it
open data sets used in the experiment
source code of all the components
environment of execution
stable references between all this
The first two items are already widely acknowledged
open access Berlin declaration, 2003
http://openaccess.mpg.de
open data sets Declaration on open access to scientific data,
OECD, 2004 http://bit.ly/1lGubJQ
What about the code?
Roberto Di Cosmo Preserving Software for Reproductibility 8/26
Why Open Source?
Having access to (all) the source of (all) the code used in an
experiment is important:
even the simplest experiments depend on a wealth of
components and configuration options
source code is like the proof used in a theorem: Fermat
statements like “the details are omitted due to lack of space”
access to all the source code is not just necessary to
reproduce, it is also useful to evolve and modify, to build new
experiments from the old ones
For science, and industry ...
Free and Open Source Software is the best choice!
Roberto Di Cosmo Preserving Software for Reproductibility 9/26
Software and reproducibility
A fundamental question
How are we doing, regarding reproducibility, in Software?
The case of Computer Systems Research
A field with Computer experts ... we have high expectations!
Christian Collberg set out to check them.
Measuring Reproducibility in Computer Systems Research
Long and detailed technical report, March 2014
http://reproducibility.cs.arizona.edu/v1/tr.pdf
Roberto Di Cosmo Preserving Software for Reproductibility 11/26
Collberg’s report from the trenches
Analysis of 613 papers
8 ACM conferences: ASPLOS’12,
CCS’12, OOPSLA’12, OSDI’12,
PLDI’12, SIGMOD’12, SOSP’11,
VLDB’12
5 journals: TACO’9, TISSEC’15,
TOCS’30, TODS’37, TOPLAS’34
all very practical oriented
The basic question
can we get the code to build and run?
The workflow
Roberto Di Cosmo Preserving Software for Reproductibility 12/26
The result
That’s a whopping 81% of non reproducible works!
Roberto Di Cosmo Preserving Software for Reproductibility 13/26
The reasons (or, “the dog ate my program”)
Many issues, nice anectotes, and it finally boils down to
Availability
Traceability
Environment
Automation
Documentation
Understanding (Open Source)
The first two are actually important software preservation issues:
yes, code can be destroyed, and we can loose trace of it!
Roberto Di Cosmo Preserving Software for Reproductibility 14/26
Like all digital information, software is fragile
An example is worth a thousand words...
Roberto Di Cosmo Preserving Software for Reproductibility 16/26
Inconsiderate or malicious loss of code
The Year 2000 Bug ... uncovered an inconvenient truth
in 1999, an estimated 40% of companies had either
lost, or thrown away the original source code for their
systems!
CodeSpaces: source code hosting, 2007-2014
Yes, for seven years all seemed ok.
No, they did not recover the data.
Roberto Di Cosmo Preserving Software for Reproductibility 17/26
Business-driven loss of code support: Google
Roberto Di Cosmo Preserving Software for Reproductibility 18/26
Business-driven loss of code support: Google, cont’d.
Roberto Di Cosmo Preserving Software for Reproductibility 19/26
Disruption of the web of reference
Web links are not permanent (even permalinks)
there is no general guarantee that a URL which at
one time points to a given object continues to do so
T. Berners-Lee et al. RFC 1738.
URLs used in articles decay!
Analysis of IEEE Computer (Computer), and the Communications
of the ACM (CACM): 1995-1999
the half-life of a referenced URL is approximately 4 years
from its publication date
D. Spinellis. The Decay and Failures of URL References.
Communications of the ACM, 46(1):71-77, January 2003.
Roberto Di Cosmo Preserving Software for Reproductibility 20/26
Preservation of digital information is on the rise
A wealth of initiatives around us
generalist the Web at archive.org; National Digital
Information Infrastructure and Preservation Program
(NDIIPP, USA); ...
culture books, music, video: INA (FR);
http://www.nationalarchives.gov.uk (UK) ...
social networks Twitter is archived by the Library of Congress!
libraries and scholarly work ArXiv; Digital Preservation Network
http://www.dpn.org/; ...
scientific data CINES (FR); Zenodo/OpenAire (CERN); ...
And yet, up to now...
software is largely ignored as an object of preservation...
computer scientists are absent from the preservation landscape!
It’s time to change all this!
Roberto Di Cosmo Preserving Software for Reproductibility 22/26
Software Preservation Challenges: it’s different!
Unlike for books or movies, there is a big difference between using
and understanding a piece of software.
Using software
Requires an executable, and access to the execution environment
Understanding software
Requires access to the source code:
The source code for a work means the preferred form
of the work for making modifications to it.
— GNU General Public Licence, version 2
For reproducibility, we need both!
Roberto Di Cosmo Preserving Software for Reproductibility 23/26
Software Preservation Challenges: it’s different!
Preserving software is more complex than archiving books or
scientific articles.
interdependencies a program relies on other software as well as
specific hardware components to be executed;
so does our understanding of its functionality
evolution software is a live object: its detailed history contains
key knowledge
To preserve software it is not enough to mimic processes that were
intended to archive books, scientific articles or data.
Roberto Di Cosmo Preserving Software for Reproductibility 24/26
The Knowledge Conservancy Magic Triangle
articles ArXiv, HAL, ...
data Zenodo, OpenAire, ...
software is the next to come, and is our mission
Roberto Di Cosmo Preserving Software for Reproductibility 25/26
Conclusions
Preservation is the foundation of all reproductibility efforts
We all must do our best to
preserve and make accessible scientific articles and data
preserve and make accessible the software
Because software embodies our scientific and technical knowledge
Conoscere `e saper fare.
Giambattista Vico, 1668–1744, Naples
Questions ?
Roberto Di Cosmo Preserving Software for Reproductibility 26/26

More Related Content

Viewers also liked

ScilabTEC 2015 - Inria
ScilabTEC 2015 - InriaScilabTEC 2015 - Inria
ScilabTEC 2015 - Inria
Scilab
 
ScilabTEC 2015 - Inmetro
ScilabTEC 2015 - InmetroScilabTEC 2015 - Inmetro
ScilabTEC 2015 - Inmetro
Scilab
 
ScilabTEC 2015 - Noesis Solutions
ScilabTEC 2015 - Noesis SolutionsScilabTEC 2015 - Noesis Solutions
ScilabTEC 2015 - Noesis Solutions
Scilab
 
ScilabTEC 2015 - Xilinx
ScilabTEC 2015 - XilinxScilabTEC 2015 - Xilinx
ScilabTEC 2015 - Xilinx
Scilab
 
ScilabTEC 2015 - J. Richalet
ScilabTEC 2015 - J. RichaletScilabTEC 2015 - J. Richalet
ScilabTEC 2015 - J. Richalet
Scilab
 
ScilabTEC 2015 - CEA/CESTA
ScilabTEC 2015 - CEA/CESTAScilabTEC 2015 - CEA/CESTA
ScilabTEC 2015 - CEA/CESTA
Scilab
 
ScilabTEC 2015 - TUM
ScilabTEC 2015 - TUMScilabTEC 2015 - TUM
ScilabTEC 2015 - TUM
Scilab
 
ScilabTEC 2015 - Bavarian Center for Agriculture
ScilabTEC 2015 - Bavarian Center for AgricultureScilabTEC 2015 - Bavarian Center for Agriculture
ScilabTEC 2015 - Bavarian Center for Agriculture
Scilab
 
ScilabTEC 2015 - KIT
ScilabTEC 2015 - KITScilabTEC 2015 - KIT
ScilabTEC 2015 - KIT
Scilab
 
ScilabTEC 2015 - CNES
ScilabTEC 2015 - CNESScilabTEC 2015 - CNES
ScilabTEC 2015 - CNES
Scilab
 
Scilab/Xcos pour l'enseignement des sciences de l'ingénieur
Scilab/Xcos pour l'enseignement des sciences de l'ingénieurScilab/Xcos pour l'enseignement des sciences de l'ingénieur
Scilab/Xcos pour l'enseignement des sciences de l'ingénieur
Scilab
 

Viewers also liked (11)

ScilabTEC 2015 - Inria
ScilabTEC 2015 - InriaScilabTEC 2015 - Inria
ScilabTEC 2015 - Inria
 
ScilabTEC 2015 - Inmetro
ScilabTEC 2015 - InmetroScilabTEC 2015 - Inmetro
ScilabTEC 2015 - Inmetro
 
ScilabTEC 2015 - Noesis Solutions
ScilabTEC 2015 - Noesis SolutionsScilabTEC 2015 - Noesis Solutions
ScilabTEC 2015 - Noesis Solutions
 
ScilabTEC 2015 - Xilinx
ScilabTEC 2015 - XilinxScilabTEC 2015 - Xilinx
ScilabTEC 2015 - Xilinx
 
ScilabTEC 2015 - J. Richalet
ScilabTEC 2015 - J. RichaletScilabTEC 2015 - J. Richalet
ScilabTEC 2015 - J. Richalet
 
ScilabTEC 2015 - CEA/CESTA
ScilabTEC 2015 - CEA/CESTAScilabTEC 2015 - CEA/CESTA
ScilabTEC 2015 - CEA/CESTA
 
ScilabTEC 2015 - TUM
ScilabTEC 2015 - TUMScilabTEC 2015 - TUM
ScilabTEC 2015 - TUM
 
ScilabTEC 2015 - Bavarian Center for Agriculture
ScilabTEC 2015 - Bavarian Center for AgricultureScilabTEC 2015 - Bavarian Center for Agriculture
ScilabTEC 2015 - Bavarian Center for Agriculture
 
ScilabTEC 2015 - KIT
ScilabTEC 2015 - KITScilabTEC 2015 - KIT
ScilabTEC 2015 - KIT
 
ScilabTEC 2015 - CNES
ScilabTEC 2015 - CNESScilabTEC 2015 - CNES
ScilabTEC 2015 - CNES
 
Scilab/Xcos pour l'enseignement des sciences de l'ingénieur
Scilab/Xcos pour l'enseignement des sciences de l'ingénieurScilab/Xcos pour l'enseignement des sciences de l'ingénieur
Scilab/Xcos pour l'enseignement des sciences de l'ingénieur
 

Similar to ScilabTEC 2015 - Irill

La préservation des logiciels: défis et opportunités pour la reproductibilité...
La préservation des logiciels: défis et opportunités pour la reproductibilité...La préservation des logiciels: défis et opportunités pour la reproductibilité...
La préservation des logiciels: défis et opportunités pour la reproductibilité...
Roberto Di Cosmo
 
Reproducible Science and Deep Software Variability
Reproducible Science and Deep Software VariabilityReproducible Science and Deep Software Variability
Reproducible Science and Deep Software Variability
University of Rennes, INSA Rennes, Inria/IRISA, CNRS
 
#OSSPARIS17 - Logiciel libre pour une science reproductible, par ROBERTO DI C...
#OSSPARIS17 - Logiciel libre pour une science reproductible, par ROBERTO DI C...#OSSPARIS17 - Logiciel libre pour une science reproductible, par ROBERTO DI C...
#OSSPARIS17 - Logiciel libre pour une science reproductible, par ROBERTO DI C...
Paris Open Source Summit
 
Observability für alle
Observability für alleObservability für alle
Observability für alle
QAware GmbH
 
Achieving Impact with Open Source Software
Achieving Impact with Open Source SoftwareAchieving Impact with Open Source Software
Achieving Impact with Open Source Software
Roberto Di Cosmo
 
Cloud Operations and Analytics: Improving Distributed Systems Reliability usi...
Cloud Operations and Analytics: Improving Distributed Systems Reliability usi...Cloud Operations and Analytics: Improving Distributed Systems Reliability usi...
Cloud Operations and Analytics: Improving Distributed Systems Reliability usi...
Jorge Cardoso
 
Introduction to Software Reverse Engineering
Introduction to Software Reverse EngineeringIntroduction to Software Reverse Engineering
Introduction to Software Reverse Engineering
Teodoro Cipresso
 
Open & reproducible research - What can we do in practice?
Open & reproducible research - What can we do in practice?Open & reproducible research - What can we do in practice?
Open & reproducible research - What can we do in practice?
Felix Z. Hoffmann
 
Software Heritage, a revolutionary infrastructure for software source code, O...
Software Heritage, a revolutionary infrastructure for software source code, O...Software Heritage, a revolutionary infrastructure for software source code, O...
Software Heritage, a revolutionary infrastructure for software source code, O...
OW2
 
Of Changes and Their History
Of Changes and Their HistoryOf Changes and Their History
Of Changes and Their History
University of Zurich
 
Icpc16.ppt
Icpc16.pptIcpc16.ppt
Icpc16.ppt
Ptidej Team
 
Icpc16.ppt
Icpc16.pptIcpc16.ppt
Open Source Software Storyboard Ver9
Open Source Software Storyboard Ver9Open Source Software Storyboard Ver9
Open Source Software Storyboard Ver9
Henry Briggs
 
An Overview Of The Singularity Project
An  Overview Of The  Singularity  ProjectAn  Overview Of The  Singularity  Project
An Overview Of The Singularity Project
alanocu
 
Using Docker Containers to Improve Reproducibility in Software and Web Engine...
Using Docker Containers to Improve Reproducibility in Software and Web Engine...Using Docker Containers to Improve Reproducibility in Software and Web Engine...
Using Docker Containers to Improve Reproducibility in Software and Web Engine...
Vincenzo Ferme
 
Tsunami of Technologies. Are we prepared?
Tsunami of Technologies. Are we prepared?Tsunami of Technologies. Are we prepared?
Tsunami of Technologies. Are we prepared?
msyukor
 
Deep Software Variability and Frictionless Reproducibility
Deep Software Variability and Frictionless ReproducibilityDeep Software Variability and Frictionless Reproducibility
Deep Software Variability and Frictionless Reproducibility
University of Rennes, INSA Rennes, Inria/IRISA, CNRS
 
Codemotion Berlin 2015 recap
Codemotion Berlin 2015   recapCodemotion Berlin 2015   recap
Codemotion Berlin 2015 recap
Torben Dohrn
 
Primers or Reminders? The Effects of Existing Review Comments on Code Review
Primers or Reminders? The Effects of Existing Review Comments on Code ReviewPrimers or Reminders? The Effects of Existing Review Comments on Code Review
Primers or Reminders? The Effects of Existing Review Comments on Code Review
Delft University of Technology
 
Go open2010 sde_20100417
Go open2010 sde_20100417Go open2010 sde_20100417
Go open2010 sde_20100417
Sandro D'Elia
 

Similar to ScilabTEC 2015 - Irill (20)

La préservation des logiciels: défis et opportunités pour la reproductibilité...
La préservation des logiciels: défis et opportunités pour la reproductibilité...La préservation des logiciels: défis et opportunités pour la reproductibilité...
La préservation des logiciels: défis et opportunités pour la reproductibilité...
 
Reproducible Science and Deep Software Variability
Reproducible Science and Deep Software VariabilityReproducible Science and Deep Software Variability
Reproducible Science and Deep Software Variability
 
#OSSPARIS17 - Logiciel libre pour une science reproductible, par ROBERTO DI C...
#OSSPARIS17 - Logiciel libre pour une science reproductible, par ROBERTO DI C...#OSSPARIS17 - Logiciel libre pour une science reproductible, par ROBERTO DI C...
#OSSPARIS17 - Logiciel libre pour une science reproductible, par ROBERTO DI C...
 
Observability für alle
Observability für alleObservability für alle
Observability für alle
 
Achieving Impact with Open Source Software
Achieving Impact with Open Source SoftwareAchieving Impact with Open Source Software
Achieving Impact with Open Source Software
 
Cloud Operations and Analytics: Improving Distributed Systems Reliability usi...
Cloud Operations and Analytics: Improving Distributed Systems Reliability usi...Cloud Operations and Analytics: Improving Distributed Systems Reliability usi...
Cloud Operations and Analytics: Improving Distributed Systems Reliability usi...
 
Introduction to Software Reverse Engineering
Introduction to Software Reverse EngineeringIntroduction to Software Reverse Engineering
Introduction to Software Reverse Engineering
 
Open & reproducible research - What can we do in practice?
Open & reproducible research - What can we do in practice?Open & reproducible research - What can we do in practice?
Open & reproducible research - What can we do in practice?
 
Software Heritage, a revolutionary infrastructure for software source code, O...
Software Heritage, a revolutionary infrastructure for software source code, O...Software Heritage, a revolutionary infrastructure for software source code, O...
Software Heritage, a revolutionary infrastructure for software source code, O...
 
Of Changes and Their History
Of Changes and Their HistoryOf Changes and Their History
Of Changes and Their History
 
Icpc16.ppt
Icpc16.pptIcpc16.ppt
Icpc16.ppt
 
Icpc16.ppt
Icpc16.pptIcpc16.ppt
Icpc16.ppt
 
Open Source Software Storyboard Ver9
Open Source Software Storyboard Ver9Open Source Software Storyboard Ver9
Open Source Software Storyboard Ver9
 
An Overview Of The Singularity Project
An  Overview Of The  Singularity  ProjectAn  Overview Of The  Singularity  Project
An Overview Of The Singularity Project
 
Using Docker Containers to Improve Reproducibility in Software and Web Engine...
Using Docker Containers to Improve Reproducibility in Software and Web Engine...Using Docker Containers to Improve Reproducibility in Software and Web Engine...
Using Docker Containers to Improve Reproducibility in Software and Web Engine...
 
Tsunami of Technologies. Are we prepared?
Tsunami of Technologies. Are we prepared?Tsunami of Technologies. Are we prepared?
Tsunami of Technologies. Are we prepared?
 
Deep Software Variability and Frictionless Reproducibility
Deep Software Variability and Frictionless ReproducibilityDeep Software Variability and Frictionless Reproducibility
Deep Software Variability and Frictionless Reproducibility
 
Codemotion Berlin 2015 recap
Codemotion Berlin 2015   recapCodemotion Berlin 2015   recap
Codemotion Berlin 2015 recap
 
Primers or Reminders? The Effects of Existing Review Comments on Code Review
Primers or Reminders? The Effects of Existing Review Comments on Code ReviewPrimers or Reminders? The Effects of Existing Review Comments on Code Review
Primers or Reminders? The Effects of Existing Review Comments on Code Review
 
Go open2010 sde_20100417
Go open2010 sde_20100417Go open2010 sde_20100417
Go open2010 sde_20100417
 

More from Scilab

Statistical Analysis for Robust Design
Statistical Analysis for Robust DesignStatistical Analysis for Robust Design
Statistical Analysis for Robust Design
Scilab
 
Electric motor optimization
Electric motor optimizationElectric motor optimization
Electric motor optimization
Scilab
 
Asteroidlanding - Scilab conference 2019 Keynote
Asteroidlanding - Scilab conference 2019 KeynoteAsteroidlanding - Scilab conference 2019 Keynote
Asteroidlanding - Scilab conference 2019 Keynote
Scilab
 
Faster Time to Market using Scilab/XCOS/X2C for motor control algorithm devel...
Faster Time to Market using Scilab/XCOS/X2C for motor control algorithm devel...Faster Time to Market using Scilab/XCOS/X2C for motor control algorithm devel...
Faster Time to Market using Scilab/XCOS/X2C for motor control algorithm devel...
Scilab
 
Scilab and Xcos for Very Low Earth Orbits satellites modelling
Scilab and Xcos for Very Low Earth Orbits satellites modellingScilab and Xcos for Very Low Earth Orbits satellites modelling
Scilab and Xcos for Very Low Earth Orbits satellites modelling
Scilab
 
X2C -a tool for model-based control development and automated code generation...
X2C -a tool for model-based control development and automated code generation...X2C -a tool for model-based control development and automated code generation...
X2C -a tool for model-based control development and automated code generation...
Scilab
 
A Real-Time Interface for Xcos – an illustrative demonstration using a batter...
A Real-Time Interface for Xcos – an illustrative demonstration using a batter...A Real-Time Interface for Xcos – an illustrative demonstration using a batter...
A Real-Time Interface for Xcos – an illustrative demonstration using a batter...
Scilab
 
Aircraft Simulation Model and Flight Control Laws Design Using Scilab and XCos
Aircraft Simulation Model and Flight Control Laws Design Using Scilab and XCosAircraft Simulation Model and Flight Control Laws Design Using Scilab and XCos
Aircraft Simulation Model and Flight Control Laws Design Using Scilab and XCos
Scilab
 
Scilab for real dummies j.heikell - part3
Scilab for real dummies j.heikell - part3Scilab for real dummies j.heikell - part3
Scilab for real dummies j.heikell - part3
Scilab
 
Scilab for real dummies j.heikell - part 2
Scilab for real dummies j.heikell - part 2Scilab for real dummies j.heikell - part 2
Scilab for real dummies j.heikell - part 2
Scilab
 
Scilab for real dummies j.heikell - part 1
Scilab for real dummies j.heikell - part 1Scilab for real dummies j.heikell - part 1
Scilab for real dummies j.heikell - part 1
Scilab
 
Multiobjective optimization and Genetic algorithms in Scilab
Multiobjective optimization and Genetic algorithms in ScilabMultiobjective optimization and Genetic algorithms in Scilab
Multiobjective optimization and Genetic algorithms in Scilab
Scilab
 
Scilab optimization workshop
Scilab optimization workshop Scilab optimization workshop
Scilab optimization workshop
Scilab
 
INRA @ Scilab Conference 2018
INRA @ Scilab Conference 2018INRA @ Scilab Conference 2018
INRA @ Scilab Conference 2018
Scilab
 
Qualcomm @ Scilab Conference 2018
Qualcomm @ Scilab Conference 2018Qualcomm @ Scilab Conference 2018
Qualcomm @ Scilab Conference 2018
Scilab
 
Sanofi @ Scilab Conference 2018
Sanofi @ Scilab Conference 2018Sanofi @ Scilab Conference 2018
Sanofi @ Scilab Conference 2018
Scilab
 
University of Applied Science Esslingen @ Scilab Conference 2018
University of Applied Science Esslingen @ Scilab Conference 2018University of Applied Science Esslingen @ Scilab Conference 2018
University of Applied Science Esslingen @ Scilab Conference 2018
Scilab
 
DLR @ Scilab Conference 2018
DLR @ Scilab Conference 2018DLR @ Scilab Conference 2018
DLR @ Scilab Conference 2018
Scilab
 
Fraunhofer IIS @ Scilab Conference 2018
Fraunhofer IIS @ Scilab Conference 2018Fraunhofer IIS @ Scilab Conference 2018
Fraunhofer IIS @ Scilab Conference 2018
Scilab
 
Arcelormittal @ Scilab Conference 2018
Arcelormittal @ Scilab Conference 2018Arcelormittal @ Scilab Conference 2018
Arcelormittal @ Scilab Conference 2018
Scilab
 

More from Scilab (20)

Statistical Analysis for Robust Design
Statistical Analysis for Robust DesignStatistical Analysis for Robust Design
Statistical Analysis for Robust Design
 
Electric motor optimization
Electric motor optimizationElectric motor optimization
Electric motor optimization
 
Asteroidlanding - Scilab conference 2019 Keynote
Asteroidlanding - Scilab conference 2019 KeynoteAsteroidlanding - Scilab conference 2019 Keynote
Asteroidlanding - Scilab conference 2019 Keynote
 
Faster Time to Market using Scilab/XCOS/X2C for motor control algorithm devel...
Faster Time to Market using Scilab/XCOS/X2C for motor control algorithm devel...Faster Time to Market using Scilab/XCOS/X2C for motor control algorithm devel...
Faster Time to Market using Scilab/XCOS/X2C for motor control algorithm devel...
 
Scilab and Xcos for Very Low Earth Orbits satellites modelling
Scilab and Xcos for Very Low Earth Orbits satellites modellingScilab and Xcos for Very Low Earth Orbits satellites modelling
Scilab and Xcos for Very Low Earth Orbits satellites modelling
 
X2C -a tool for model-based control development and automated code generation...
X2C -a tool for model-based control development and automated code generation...X2C -a tool for model-based control development and automated code generation...
X2C -a tool for model-based control development and automated code generation...
 
A Real-Time Interface for Xcos – an illustrative demonstration using a batter...
A Real-Time Interface for Xcos – an illustrative demonstration using a batter...A Real-Time Interface for Xcos – an illustrative demonstration using a batter...
A Real-Time Interface for Xcos – an illustrative demonstration using a batter...
 
Aircraft Simulation Model and Flight Control Laws Design Using Scilab and XCos
Aircraft Simulation Model and Flight Control Laws Design Using Scilab and XCosAircraft Simulation Model and Flight Control Laws Design Using Scilab and XCos
Aircraft Simulation Model and Flight Control Laws Design Using Scilab and XCos
 
Scilab for real dummies j.heikell - part3
Scilab for real dummies j.heikell - part3Scilab for real dummies j.heikell - part3
Scilab for real dummies j.heikell - part3
 
Scilab for real dummies j.heikell - part 2
Scilab for real dummies j.heikell - part 2Scilab for real dummies j.heikell - part 2
Scilab for real dummies j.heikell - part 2
 
Scilab for real dummies j.heikell - part 1
Scilab for real dummies j.heikell - part 1Scilab for real dummies j.heikell - part 1
Scilab for real dummies j.heikell - part 1
 
Multiobjective optimization and Genetic algorithms in Scilab
Multiobjective optimization and Genetic algorithms in ScilabMultiobjective optimization and Genetic algorithms in Scilab
Multiobjective optimization and Genetic algorithms in Scilab
 
Scilab optimization workshop
Scilab optimization workshop Scilab optimization workshop
Scilab optimization workshop
 
INRA @ Scilab Conference 2018
INRA @ Scilab Conference 2018INRA @ Scilab Conference 2018
INRA @ Scilab Conference 2018
 
Qualcomm @ Scilab Conference 2018
Qualcomm @ Scilab Conference 2018Qualcomm @ Scilab Conference 2018
Qualcomm @ Scilab Conference 2018
 
Sanofi @ Scilab Conference 2018
Sanofi @ Scilab Conference 2018Sanofi @ Scilab Conference 2018
Sanofi @ Scilab Conference 2018
 
University of Applied Science Esslingen @ Scilab Conference 2018
University of Applied Science Esslingen @ Scilab Conference 2018University of Applied Science Esslingen @ Scilab Conference 2018
University of Applied Science Esslingen @ Scilab Conference 2018
 
DLR @ Scilab Conference 2018
DLR @ Scilab Conference 2018DLR @ Scilab Conference 2018
DLR @ Scilab Conference 2018
 
Fraunhofer IIS @ Scilab Conference 2018
Fraunhofer IIS @ Scilab Conference 2018Fraunhofer IIS @ Scilab Conference 2018
Fraunhofer IIS @ Scilab Conference 2018
 
Arcelormittal @ Scilab Conference 2018
Arcelormittal @ Scilab Conference 2018Arcelormittal @ Scilab Conference 2018
Arcelormittal @ Scilab Conference 2018
 

Recently uploaded

A study on drug utilization evaluation of bronchodilators using DDD method
A study on drug utilization evaluation of bronchodilators using DDD methodA study on drug utilization evaluation of bronchodilators using DDD method
A study on drug utilization evaluation of bronchodilators using DDD method
Dr. Chihiro
 
Lucknow @Girls @ℂall Gomti Nagar 08630512678 @Girls @ℂall Service
Lucknow @Girls @ℂall  Gomti Nagar 08630512678  @Girls @ℂall ServiceLucknow @Girls @ℂall  Gomti Nagar 08630512678  @Girls @ℂall Service
Lucknow @Girls @ℂall Gomti Nagar 08630512678 @Girls @ℂall Service
veenita788
 
MINI PROJECT REPORT about the multidiseases.pdf
MINI PROJECT REPORT about the multidiseases.pdfMINI PROJECT REPORT about the multidiseases.pdf
MINI PROJECT REPORT about the multidiseases.pdf
BECHAADITYA
 
Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...
Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...
Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...
parichopra4
 
ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...
ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...
ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...
DrAdoGarba
 
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
rawankhanlove256
 
UW biyezheng degree offer diploma Transcript
UW biyezheng degree offer diploma TranscriptUW biyezheng degree offer diploma Transcript
UW biyezheng degree offer diploma Transcript
xmevus
 
stackconf 2024 | Generative AI Security — A Practical Guide to Securing Your ...
stackconf 2024 | Generative AI Security — A Practical Guide to Securing Your ...stackconf 2024 | Generative AI Security — A Practical Guide to Securing Your ...
stackconf 2024 | Generative AI Security — A Practical Guide to Securing Your ...
NETWAYS
 
Colorfcul Presentation - Public Relations
Colorfcul Presentation - Public RelationsColorfcul Presentation - Public Relations
Colorfcul Presentation - Public Relations
StephanieFeliciano8
 
UMiami biyezheng degree offer diploma Transcript
UMiami biyezheng degree offer diploma TranscriptUMiami biyezheng degree offer diploma Transcript
UMiami biyezheng degree offer diploma Transcript
xmevus
 
VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...
VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...
VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...
saroohilakhatariroy
 
Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...
Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...
Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...
kishanaaani
 
Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...
Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...
Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...
sankisogandhi
 
@ℂall Lucknow @Girls Fazullaganj 08630512678 @Girls Service @ℂall
@ℂall Lucknow @Girls Fazullaganj 08630512678  @Girls Service @ℂall@ℂall Lucknow @Girls Fazullaganj 08630512678  @Girls Service @ℂall
@ℂall Lucknow @Girls Fazullaganj 08630512678 @Girls Service @ℂall
veenita788
 
VIP Ahmedabad Girls Call Ahmedabad 0X0000000X Doorstep High-Profile Girl Serv...
VIP Ahmedabad Girls Call Ahmedabad 0X0000000X Doorstep High-Profile Girl Serv...VIP Ahmedabad Girls Call Ahmedabad 0X0000000X Doorstep High-Profile Girl Serv...
VIP Ahmedabad Girls Call Ahmedabad 0X0000000X Doorstep High-Profile Girl Serv...
satpalsheravatmumbai
 
Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...
Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...
Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...
seenaoberoi
 
Lucknow Girls Call Fazullaganj 08630512678 Provide Best And Top Girl Service ...
Lucknow Girls Call Fazullaganj 08630512678 Provide Best And Top Girl Service ...Lucknow Girls Call Fazullaganj 08630512678 Provide Best And Top Girl Service ...
Lucknow Girls Call Fazullaganj 08630512678 Provide Best And Top Girl Service ...
bangaloreakshitakaus
 
Call India - AmanTel on the App Store.ppt
Call India - AmanTel on the App Store.pptCall India - AmanTel on the App Store.ppt
Call India - AmanTel on the App Store.ppt
Best International calling app on the market
 
GT biyezheng degree offer diploma Transcript
GT biyezheng degree offer diploma TranscriptGT biyezheng degree offer diploma Transcript
GT biyezheng degree offer diploma Transcript
xmevus
 
UCI biyezheng degree offer diploma Transcript
UCI biyezheng degree offer diploma TranscriptUCI biyezheng degree offer diploma Transcript
UCI biyezheng degree offer diploma Transcript
xmevus
 

Recently uploaded (20)

A study on drug utilization evaluation of bronchodilators using DDD method
A study on drug utilization evaluation of bronchodilators using DDD methodA study on drug utilization evaluation of bronchodilators using DDD method
A study on drug utilization evaluation of bronchodilators using DDD method
 
Lucknow @Girls @ℂall Gomti Nagar 08630512678 @Girls @ℂall Service
Lucknow @Girls @ℂall  Gomti Nagar 08630512678  @Girls @ℂall ServiceLucknow @Girls @ℂall  Gomti Nagar 08630512678  @Girls @ℂall Service
Lucknow @Girls @ℂall Gomti Nagar 08630512678 @Girls @ℂall Service
 
MINI PROJECT REPORT about the multidiseases.pdf
MINI PROJECT REPORT about the multidiseases.pdfMINI PROJECT REPORT about the multidiseases.pdf
MINI PROJECT REPORT about the multidiseases.pdf
 
Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...
Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...
Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...
 
ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...
ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...
ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...
 
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
 
UW biyezheng degree offer diploma Transcript
UW biyezheng degree offer diploma TranscriptUW biyezheng degree offer diploma Transcript
UW biyezheng degree offer diploma Transcript
 
stackconf 2024 | Generative AI Security — A Practical Guide to Securing Your ...
stackconf 2024 | Generative AI Security — A Practical Guide to Securing Your ...stackconf 2024 | Generative AI Security — A Practical Guide to Securing Your ...
stackconf 2024 | Generative AI Security — A Practical Guide to Securing Your ...
 
Colorfcul Presentation - Public Relations
Colorfcul Presentation - Public RelationsColorfcul Presentation - Public Relations
Colorfcul Presentation - Public Relations
 
UMiami biyezheng degree offer diploma Transcript
UMiami biyezheng degree offer diploma TranscriptUMiami biyezheng degree offer diploma Transcript
UMiami biyezheng degree offer diploma Transcript
 
VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...
VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...
VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...
 
Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...
Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...
Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...
 
Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...
Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...
Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...
 
@ℂall Lucknow @Girls Fazullaganj 08630512678 @Girls Service @ℂall
@ℂall Lucknow @Girls Fazullaganj 08630512678  @Girls Service @ℂall@ℂall Lucknow @Girls Fazullaganj 08630512678  @Girls Service @ℂall
@ℂall Lucknow @Girls Fazullaganj 08630512678 @Girls Service @ℂall
 
VIP Ahmedabad Girls Call Ahmedabad 0X0000000X Doorstep High-Profile Girl Serv...
VIP Ahmedabad Girls Call Ahmedabad 0X0000000X Doorstep High-Profile Girl Serv...VIP Ahmedabad Girls Call Ahmedabad 0X0000000X Doorstep High-Profile Girl Serv...
VIP Ahmedabad Girls Call Ahmedabad 0X0000000X Doorstep High-Profile Girl Serv...
 
Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...
Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...
Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...
 
Lucknow Girls Call Fazullaganj 08630512678 Provide Best And Top Girl Service ...
Lucknow Girls Call Fazullaganj 08630512678 Provide Best And Top Girl Service ...Lucknow Girls Call Fazullaganj 08630512678 Provide Best And Top Girl Service ...
Lucknow Girls Call Fazullaganj 08630512678 Provide Best And Top Girl Service ...
 
Call India - AmanTel on the App Store.ppt
Call India - AmanTel on the App Store.pptCall India - AmanTel on the App Store.ppt
Call India - AmanTel on the App Store.ppt
 
GT biyezheng degree offer diploma Transcript
GT biyezheng degree offer diploma TranscriptGT biyezheng degree offer diploma Transcript
GT biyezheng degree offer diploma Transcript
 
UCI biyezheng degree offer diploma Transcript
UCI biyezheng degree offer diploma TranscriptUCI biyezheng degree offer diploma Transcript
UCI biyezheng degree offer diploma Transcript
 

ScilabTEC 2015 - Irill

  • 1. Preserving Software Challenges and Opportunities for Reproducibility of Science and Technology Roberto Di Cosmo Scilabtec 2015 - Paris Irill - INRIA - University Paris Diderot roberto@dicosmo.org 21st of May 2015 Roberto Di Cosmo Preserving Software for Reproductibility 1/26
  • 2. How we built our scientific knowledge The experimental method make an observation formulate an hypothesis set up an experiment formulate a theory And then we reproduce and verify. Reproducibility is the key non-reproducible single occurrences are of no significance to science Karl Popper, The Logic of Scientific Discovery, 1934 Roberto Di Cosmo Preserving Software for Reproductibility 3/26
  • 3. Reproducibility, today Reproducibility (Wikipedia) the ability of an entire experiment or study to be reproduced, either by the researcher or by someone else working independently. It is one of the main principles of the scientific method. Why we want it foundation of the scientific method accelerator of research: allows to build upon previous work visibility reproducible results are cited more often Piwowar et al. Sharing Detailed Research Data Is Associated with Increased Citation Rate. PLoS ONE 2(3): e308, 2007 transparency of results eases acceptance necessary for industrial transfer reproducibility is the essence of industry! Roberto Di Cosmo Preserving Software for Reproductibility 4/26
  • 4. Today: Software is Science’s cornerstone Software is an essential component of modern scientific research Deep knowledge from all fields is embodied in complex software systems Top 100 papers (Nature, October 2014) [...] the vast majority describe experimen- tal methods or software that have become essential in their fields. http: // www. nature. com/ news/ the-top-100-papers-1. 16224 Software is the pillar of science Roberto Di Cosmo Preserving Software for Reproductibility 6/26
  • 5. Software is Pervasive At the heart of technology televisions/fridges ≈ 10M SLOC phones ≈ 20M SLOC cars ≈ 100M SLOC IoT ... Key mediator for accessing all information (c) Banski Information is a main pillar of our modern societies. Absent an ability to correctly interpret dig- ital information, we are left with [...] “rot- ting bits” [...] of no value. Vinton G. Cerf IEEE 2011 Software is an essential enabler for all technology Roberto Di Cosmo Preserving Software for Reproductibility 7/26
  • 6. Reproducibility in the digital age For an experiment involving software, we need open access to the scientific article describing it open data sets used in the experiment source code of all the components environment of execution stable references between all this The first two items are already widely acknowledged open access Berlin declaration, 2003 http://openaccess.mpg.de open data sets Declaration on open access to scientific data, OECD, 2004 http://bit.ly/1lGubJQ What about the code? Roberto Di Cosmo Preserving Software for Reproductibility 8/26
  • 7. Why Open Source? Having access to (all) the source of (all) the code used in an experiment is important: even the simplest experiments depend on a wealth of components and configuration options source code is like the proof used in a theorem: Fermat statements like “the details are omitted due to lack of space” access to all the source code is not just necessary to reproduce, it is also useful to evolve and modify, to build new experiments from the old ones For science, and industry ... Free and Open Source Software is the best choice! Roberto Di Cosmo Preserving Software for Reproductibility 9/26
  • 8. Software and reproducibility A fundamental question How are we doing, regarding reproducibility, in Software? The case of Computer Systems Research A field with Computer experts ... we have high expectations! Christian Collberg set out to check them. Measuring Reproducibility in Computer Systems Research Long and detailed technical report, March 2014 http://reproducibility.cs.arizona.edu/v1/tr.pdf Roberto Di Cosmo Preserving Software for Reproductibility 11/26
  • 9. Collberg’s report from the trenches Analysis of 613 papers 8 ACM conferences: ASPLOS’12, CCS’12, OOPSLA’12, OSDI’12, PLDI’12, SIGMOD’12, SOSP’11, VLDB’12 5 journals: TACO’9, TISSEC’15, TOCS’30, TODS’37, TOPLAS’34 all very practical oriented The basic question can we get the code to build and run? The workflow Roberto Di Cosmo Preserving Software for Reproductibility 12/26
  • 10. The result That’s a whopping 81% of non reproducible works! Roberto Di Cosmo Preserving Software for Reproductibility 13/26
  • 11. The reasons (or, “the dog ate my program”) Many issues, nice anectotes, and it finally boils down to Availability Traceability Environment Automation Documentation Understanding (Open Source) The first two are actually important software preservation issues: yes, code can be destroyed, and we can loose trace of it! Roberto Di Cosmo Preserving Software for Reproductibility 14/26
  • 12. Like all digital information, software is fragile An example is worth a thousand words... Roberto Di Cosmo Preserving Software for Reproductibility 16/26
  • 13. Inconsiderate or malicious loss of code The Year 2000 Bug ... uncovered an inconvenient truth in 1999, an estimated 40% of companies had either lost, or thrown away the original source code for their systems! CodeSpaces: source code hosting, 2007-2014 Yes, for seven years all seemed ok. No, they did not recover the data. Roberto Di Cosmo Preserving Software for Reproductibility 17/26
  • 14. Business-driven loss of code support: Google Roberto Di Cosmo Preserving Software for Reproductibility 18/26
  • 15. Business-driven loss of code support: Google, cont’d. Roberto Di Cosmo Preserving Software for Reproductibility 19/26
  • 16. Disruption of the web of reference Web links are not permanent (even permalinks) there is no general guarantee that a URL which at one time points to a given object continues to do so T. Berners-Lee et al. RFC 1738. URLs used in articles decay! Analysis of IEEE Computer (Computer), and the Communications of the ACM (CACM): 1995-1999 the half-life of a referenced URL is approximately 4 years from its publication date D. Spinellis. The Decay and Failures of URL References. Communications of the ACM, 46(1):71-77, January 2003. Roberto Di Cosmo Preserving Software for Reproductibility 20/26
  • 17. Preservation of digital information is on the rise A wealth of initiatives around us generalist the Web at archive.org; National Digital Information Infrastructure and Preservation Program (NDIIPP, USA); ... culture books, music, video: INA (FR); http://www.nationalarchives.gov.uk (UK) ... social networks Twitter is archived by the Library of Congress! libraries and scholarly work ArXiv; Digital Preservation Network http://www.dpn.org/; ... scientific data CINES (FR); Zenodo/OpenAire (CERN); ... And yet, up to now... software is largely ignored as an object of preservation... computer scientists are absent from the preservation landscape! It’s time to change all this! Roberto Di Cosmo Preserving Software for Reproductibility 22/26
  • 18. Software Preservation Challenges: it’s different! Unlike for books or movies, there is a big difference between using and understanding a piece of software. Using software Requires an executable, and access to the execution environment Understanding software Requires access to the source code: The source code for a work means the preferred form of the work for making modifications to it. — GNU General Public Licence, version 2 For reproducibility, we need both! Roberto Di Cosmo Preserving Software for Reproductibility 23/26
  • 19. Software Preservation Challenges: it’s different! Preserving software is more complex than archiving books or scientific articles. interdependencies a program relies on other software as well as specific hardware components to be executed; so does our understanding of its functionality evolution software is a live object: its detailed history contains key knowledge To preserve software it is not enough to mimic processes that were intended to archive books, scientific articles or data. Roberto Di Cosmo Preserving Software for Reproductibility 24/26
  • 20. The Knowledge Conservancy Magic Triangle articles ArXiv, HAL, ... data Zenodo, OpenAire, ... software is the next to come, and is our mission Roberto Di Cosmo Preserving Software for Reproductibility 25/26
  • 21. Conclusions Preservation is the foundation of all reproductibility efforts We all must do our best to preserve and make accessible scientific articles and data preserve and make accessible the software Because software embodies our scientific and technical knowledge Conoscere `e saper fare. Giambattista Vico, 1668–1744, Naples Questions ? Roberto Di Cosmo Preserving Software for Reproductibility 26/26