Trustworthy results require reproducibility. Publishing code is necessary but not sufficient for complete reproducibility. Complex programs often depend on external code. This talk will introduce you to several ways to manage your software dependencies and data to pass reproducible software environments to peer-reviewers. Talk given at PyOhio 2018.07.28 by Dana Walker and Bihan Zhang

1. @pulpproj
Reproducible Environments
for Reproducible Results
Bihan Zhang
Dana Walker

2. @pulpproj
Reproducibility
Data
Code
Environment

3. @pulpproj
The Problem

4. @pulpproj
Same Code ⇏ Same Results
Developer Quality Assurance
This isn’t working :(
Works fine on my machine…

5. @pulpproj
Software Depends on Other Software
$ pip3 install jupyter graphviz pipdeptree

6. @pulpproj
Packaging

7. @pulpproj@pulpproj
Approach 1: Monolithic Packages
my_codescipy
numpy
pygame
matplotlib

8. @pulpproj@pulpproj
Approach 1: Monolithic Packages
● Easy to install
● No chance of
dependency conflicts
● Large packages
● Difficult to update &
maintain
● Opaque

9. @pulpproj@pulpproj
Approach 2: Modular Packages
my_code
pygame matplotlib
scipy
numpy

10. @pulpproj@pulpproj
Approach 2: Modular Packages
● Easy to maintain
● Transparent
● Install complexity
● Potentially unstable
updates
● Dependency Hell

11. @pulpproj
Python Packaging Ecosystem

12. @pulpproj
An Overview of the Python Ecosystem
PyPI

13. @pulpproj
Virtual Environments

14. @pulpproj
Pipenv
● virtualenv
● pip
● Pipfile

15. @pulpproj
An Overview of the Python Ecosystem
● Modular packages
● Python Package Index (PyPI)
○ Community contributed packages
○ Non-curated
● Package Manager (pip)
○ Solves dependencies at install time
○ Interface with PyPI
● Virtual Environments
○ Isolated environments
○ Allows different Python runtimes

16. @pulpproj
Development: Innovation by Default
my_code
pygame matplotlib
scipy
numpyInnovation
Stability

17. @pulpproj

18. @pulpproj

19. @pulpproj
Production: Stability
by Default
Stability
Innovation
Innovation
Stability

20. @pulpproj

21. @pulpproj

22. @pulpproj
Post-Stability Struggles
● How do you distribute your code?
● What if you need an updated dependency?
● How do you guarantee the permanence of your dependencies
StabilityInnovation

23. @pulpproj
An Overview of the Python Ecosystem
- pip
- PyPI
- venv
- pipenv

24. @pulpproj@pulpproj
Python Workflows: requirements.txt
- Transparent
- Easy to share
- Modular updates

25. @pulpproj@pulpproj
Python Workflows: pipfile, pipfile.lock
- Separates what you need and
what you have
-

26. @pulpproj
Hello Pulp!

27. @pulpproj
● Ansible Roles
● Container Images
● Generic Files
● Python Packages
Pulp manages software packages

28. @pulpproj
● RPM
● Debian
● OSTree
● Puppet
● Maven (in progress)
● Other Package Types

29. @pulpproj
Pulp organizes these software packages into Repositories

30. @pulpproj
+ =

31. @pulpproj
+ Metadata =
Publication

32. @pulpproj

33. @pulpproj@pulpproj
StabilityInnovation

34. @pulpproj
Curated Repositories

35. @pulpproj

36. @pulpproj
Promotion

37. @pulpproj
Rollback

38. @pulpproj
Other Use Cases for Pulp
● Local mirror
● Whitelisted index
● Distributing your own software

39. @pulpproj
Other Repository Management Tools
● Artifactory
● Nexus

40. @pulpproj
Deeper Reproducibility

41. @pulpproj
Python Environment
Fundamental Python Packaging Limitation
System Environment
openssl glibc

42. @pulpproj
Virtual Machines
● The ultimate monolithic package
Python Environment
System Environment

43. @pulpproj
Ansible
● Automate anything!
Python Environment
glibc

44. @pulpproj

45. @pulpproj
Ansible
● Automate anything!
○ Install system dependencies
○ Set up system config
○ Install + configure your package
○ Install Python deps
● Ansible Galaxy
● Workflows?
○ Pulp!

46. @pulpproj

47. @pulpproj
Containerization
● Immutable System Image
● How do you manage your container images?
○ Dockerfile (Transparency)
○ DockerHub (Sharing)

48. @pulpproj

49. @pulpproj
Containerization
● Immutable System Image
● How do you manage your container images?
○ Dockerfile (Transparency)
○ DockerHub (Sharing)
● Workflows?
○ Pulp!

50. @pulpproj
docker pull

51. @pulpproj
Summary
● Balances stability and innovation
○ Requirements.txt
○ Pipfile.lock
○ Ansible
○ Containers
○ Pulp!

52. @pulpproj
Questions?
● pulpproject.org
● Twitter
○ @pulpproj
○ @CodeHeeler
● Catch us in the hallway after this talk!
● Link to Slideshare: https://bit.ly/2LgqFFq