Lots of research is incomparable.
EXECUTION
REPORTING
Gathered: scattered across different repositories/catalogues
Availability of dependencies: Know and have all necessary elements available, accessible, maybe open
Change management: Data? Services? Methods? Prevent, Detect, Repair.
Execution and Making Environments: Skills/Infrastructure to run it: Portability and the Execution Platform (which can be people…), authoring and reading
Description: Explicit: How, Why, What, Where, Who, When, Comprehensive: Just Enough, Comprehensible: Independent understanding
Purpose for doing it: reason and reward sensitivity
Reporting and Preserving
SOPs for methods
Current work on research reproducibility has focused on the creation of tools for packaging research artefacts such as data and software so that the analyses can be run by others
the creation of domain-specific guidelines and checklists for the reporting of research.
How
Open software (inspection) reproduce
Closed software (execution, but not inspection) – VM! replication
FAIR Model
Description left to right
Portability up and down
FAIRport* ReproducibilityFind, Access, Interoperate, Reuse, PortPreservation - Lots of copies keeps stuff safe
Stability dimension
Add two more dimensions to our classification of themes
A virtual machine (VM) is a software implementation of a machine (i.e. a computer) that executes programs like a physical machine. Virtual machines are separated into two major classifications, based on their use and degree of correspondence to any real machine: System
Overlap of course
Static vs dynamic.
GRANULARITY
This model for audit and target of your systems
overcoming data type silos
public integrative data sets
transparency matters
cloud
Recomputation.org
Reproducibility by ExecutionRun It
Reproducibility by InspectionRead It
Availability – coverage
Gathered: scattered across resources, across the paper and supplementary materials
Availability of dependencies: Know and have all necessary elements
Change management: Data? Services? Methods? Prevent, Detect, Repair.
Execution and Making Environments: Skills/Infrastructure to run it: Portability and the Execution Platform (which can be people…), Skills/Infrastructure for authoring and reading
Description: Explicit: How, Why, What, Where, Who, When, Comprehensive: Just Enough, Comprehensible: Independent understanding
Documentation vs Bits (VMs) reproducibility
Learn/understand (reproduce and validate, reproduce using different codes) vs Run (reuse, validate, repeat, reproduce under different configs/settings)