An introduction to software maintenance in the open source context, focusing on software and human quality controls, oriented towards . Presented in July 2019 by Joel Nothman for an internal cross-skilling session in the Sydney Informatics Hub, Core Research Facilities, The University of Sydney.

1. The University of Sydney Page 1
Maintaining & Releasing
Open-Source Software
Joel Nothman
Sydney Informatics Hub

2. The University of Sydney Page 2
Different perspectives
What users see
- Packages to download
- API reference in favourite IDE
- Documentation on web site
- Usage examples
- Errors and warnings
- Change logs
- StackOverflow
What maintainers see
- Repository with development and
release branches
- Issues to resolve
- Patches (= pull requests)
- Quality assurance mechanisms
- Release cycle & roadmap
- Pool of contributors

3. The University of Sydney Page 3
This talk should encourage you to:
– think of software delivery beyond solving a task
– understand what software maintainers do
– appreciate tools that can help make code trustworthy
– evaluate software critically according to how well it is
maintained
– consider open-source software contribution for personal
development

4. The University of Sydney Page 4
The user becomes a contributor
– an error or unclarity in the
documentation
– a bug in the software
– an inefficiency in the
software
– new feature they would
like to see
– an inconsistency in the API
do nothing
fix it and keep for oneself
ask the web for a workaround
open a new issue
comment on an existing issue
offer a fix via pull request

5. The University of Sydney Page 5
The (idealised) issue lifecycle
Issue
opened
Triage
Issue
assigned
Patch
committed
Pull
request
opened
Patch
tested and
reviewed
PR
approved
and
merged
Do we agree this is a new issue?
Who is capable of solving it?
Which release should it belong to?
More
commits
Is this the right fix?
Does it work?
Could it break user code?
Is it usable?
Is it maintainable?
Commits
batched
for
release

6. The University of Sydney Page 6
What makes software good?
– Software controls
– Test suite
– Coverage measures
– Linting
– Commit history
– Human controls
– Design and Roadmap
– Review
– Standards
– Documentation
– Governance
– Human testing
– Release versioning
– Feedback!
Goals:
usable,
maintainable,
trustworthy
Credit: gleonhard@flickr

7. The University of Sydney Page 7
Software controls

8. The University of Sydney Page 8
Continuous integration is just a command
– Triggered by repo activity
or a schedule
– Set up some environment
– Run some command and
report if it succeeded
– A Travis CI config:
language: generic
install:
- test -x $HOME/miniconda/bin/conda || cmd to install miniconda
- conda create -q -n test-env $CONDA_DEPS $FIXED_CONDA_DEPS
- conda activate test-environment
- python setup.py install
script:
- pytest
- flake8 # linting code as well
after_success:
- coveralls
env:
global:
- FIXED_CONDA_DEPS="flake8 pytest pytest-cov<2.6 coveralls"
matrix:
include:
- env: CONDA_DEPS="python=2.7 pandas matplotlib"
- env: CONDA_DEPS="python=3.5 pandas=0.20 matplotlib=2.0.2
numpy=1.9.3"
- env: CONDA_DEPS="python=3.6 pandas matplotlib seaborn"

9. The University of Sydney Page 9
Check status
which commit is
being checked?
each check
provides a status
when completed
checks for
different
purposes or
environments
view the
console logs
to diagnose
checks may fail for
arbitrary reasons,
e.g. server outage

10. The University of Sydney Page 10
Continuous integration and deployment
– For each commit to a feature branch / pull request
– Checks code integrity (compile it) and quality (lint it)
– Runs tests on various platforms
– Checks test coverage is not decreasing
– Compiles the documentation for visual testing
Ø Guides whether the new work is acceptable to merge
– Periodically scheduled for release branches
– Ensures compatibility with a changing ecosystem
– Runs slow tests or platforms
– Releases nightly builds
– Updates the documentation web site

11. The University of Sydney Page 11
CI Services
– A traditional solution: Jenkins
– Hosted, free for open-source on github.com, bitbucket, etc.:
– Azure CI
– Travis CI
– Circle CI
– …
– A catalogue: https://github.com/ligurio/awesome-ci

12. The University of Sydney Page 12
Automated testing ensures that:
– The tested behaviour is correct
– The tested behaviour is stable
– across runs
– across platforms
– over time
– Changing the code hasn’t affected existing functionality
– The API is reasonably usable (?)
– It also:
– gives you confidence to make changes
– helps you structure and develop your code & specifications

13. The University of Sydney Page 13
Code coverage
– Proportion of code lines executed during tests
– Testing tools report line coverage to a centralised tool
– Each patch is rated:
• is all new code covered?
• are any old lines no longer covered?
– Measuring coverage ensures that
– The test suite not missing some code paths unintentionally
– In a non-compiled language: No obvious typos exist
– Maintainers still need to ensure logical coverage

14. The University of Sydney Page 14
Automated linting ensures that:
– Code style is consistent with a standard
– Documentation style is consistent with a standard
– Common errors aren’t overlooked
– Unused imports
– Variable name clashes
– …
– See PyCQA (pycodestyle, pydocstyle, etc.)
– See lgtm.com and others defining more advanced alerts

15. The University of Sydney Page 15
Human controls

16. The University of Sydney Page 16
Governance
– How do changes get approved?
– Stratify by risk:
– Lazy consensus: no -1
Minor documentation changes lazy consensus: 1 core developer
Features, enhancements, fixes and rewrites lazy consensus: 2 core developers
1 day to vote if possibly contentious
API principles 2/3 of core dev votes, escalated to TC
1 month to vote
New core devs lazy consensus among core devs
1 week to vote
Changes to Tech Committee (TC) 2/3 of core dev votes, ½ of TC
Lazy consensus speech acts
• +1 : approve of an idea
• LGTM: approve of the change
• +0 : ambivalently accept
• -0 : unsupportive
• -1 : veto in its current form

17. The University of Sydney Page 17
Standards and Culture
– Code style and structure
– Testing
– Documentation
– Software/platform requirements
– Review quality
– Release schedule
– Conduct
– These are mostly implied rather than explicit standards

18. The University of Sydney Page 18
Licensing
– How can someone use/distribute/modify the software?
– What code can someone include in the software?
– Scikit-learn uses BSD 3-clause licence
– Like MIT, Apache, etc., it is liberal about how the code is used.
– But requires the license and copyright to remain attached.
– And that the name of the project not be misused (“trademark”).
– E.g. GPL-licensed code can only be copied into a GPL project
– If we suspect code has been ported from CRAN, which commonly uses
GPL, we cannot include it s(without its authors’ permission).
– While BSD code can be added to a GPL project (“GPL-compatible”)

19. The University of Sydney Page 19
Design and Roadmap
– Defining the scope for a project can be very helpful
– Or at least proscribing some things that are outside of the scope
– It can be hard to balance:
– Fixing broken things
– Doing things that seem hot right now
– Adding features that have been long in the works
– Adding features that require design and hard decisions
– A roadmap should help define the essential goals
– … not merely governed by who is noisiest on the issue tracker

20. The University of Sydney Page 20
Human review
– Conversation between contributor and maintainers/community,
with iterative improvements committed by the contributor.
– The contributor should prove value.
– The maintainer should assess cost/benefit.
– Do we want this?
– Are we be able to maintain it?
– Does it do what it claims?
– Is it as easy to use as possible?
– Is it consistent with our design and standards?

21. The University of Sydney Page 21
Funding and Resourcing
(In a nutshell)
– It can be hard to get funding for OSS development
– When it is a team project, it may also be hard to use the
funding received!
– Sprint funding is helpful to bring contributors together
– Several Scientific Python projects have ongoing funding
– And Python Software Foundation supports events, coordinates Google
Summer of Code activity, etc.
– Funding bodies may or may not have a big say on
development priorities

22. The University of Sydney Page 22
The real human side
– Who contributes?
– PhD students are the traditional Scientific Python contributors
– They usually disappear after their submission
– Do contributors maintain the code they contribute?
– As a rule: no.
– Burn-out and turnover
– How do you maintain fresh blood?
– Code of conduct and equal opportunity…
– Has been hard to get long-term involvement from women
– WIMLDS has tried to create opportunities

23. The University of Sydney Page 23
Releasing with Python
and Git

24. The University of Sydney Page 24
Semantic Versioning
– A new release after 0.6 would be:
– 1.0 if there are incompatible API changes = MAJOR
– 0.7 if there are new backwards-compatible features (or new
deprecations) = MINOR
– 0.6.1 if there are backwards-compatible bug fixes = PATCH

25. The University of Sydney Page 25
Branches and Releases in Scikit-learn
Releases:
– Aim for feature releases every 6-
9 months
– Patch releases incorporate:
– Fixes to packaging errors
– Compatibility fixes
– Fixes to regressions
– Severe fixes to new
functionality
Branches:
– master branch for current
development version (e.g.
0.22dev0)
– 0.20.X, 0.21.X, etc. for adding
patches to released versions
– Features are merged into master
– Fixes are merged into master and
cherry-picked into a release
branch

26. The University of Sydney Page 26
Metadata for package managers (e.g. pip)
– Make sure setup.py & setup.cfg contain correct metadata
– You can upload pre-releases to PyPI to check rendering
– E.g. my-package 0.1rc1
– Metadata includes the release version
– In the release branch (0.21.X):
version = 0.21rc1 then 0.21.0 then 0.21.1
– In development branch (master):
version = 0.22dev0
A popular Python
convention stores
version in
mypkg.__version__
and copies this into
setup.py metadata

27. The University of Sydney Page 27
Releasing
– Tag the release commit in GitHub
– git tag 0.21.1; git push --tags
– Need to release:
– Source code
• python setup.py sdist
– Binaries (if you’ve got C/CPP/Cython extensions)
• on each platform: python setup.py bdist_wheel
• Conda build requires meta.yaml build script
– Run tests for each release
– Use twine to upload to PyPI
A tag permanently labels
a version of the code

28. The University of Sydney Page 28
Releasing Libraries vs Apps
– Releasing a library?
– Installable with pip
– Development focus is on developer experience
– Quality assurance through unit tests, runnable documentation, feedback
– Releasing an app?
– Installable with docker, etc.
– Integrates multiple libraries
– Development focus is on user experience
– Quality assurance through integration testing, user testing

29. The University of Sydney Page 29
Getting involved

30. The University of Sydney Page 30
A user can learn a lot from contributing
– Proving the value of some feature
– Efficient, idiomatic code
– How to test and document
– API design and maintenance /
deprecation
– Advanced, review-friendly Git
and GitHub skills
– Continuous integration (CI) tools
Look for these labels in GitHub
https://github.com/topics/help-wanted?l=python
https://github.com/topics/good-first-issue?l=python
Also look for projects with a
Contributors’ Guide.

31. The University of Sydney Page 31
An OSS Python library should have
✔✔✔ Source code under version control
✔✔✔ Installation with pip (using setup.py or pyproject.toml)
✔✔✔ Machine-readable dependencies
✔✔✔ Docstrings for all public API
✔✔✔ A licence
✔✔ Tests run by continuous integration
✔✔ High test coverage
✔✔ A version number Ideally in: setup.py; module.__version__; a git tag upon release
✔✔ A public issue tracker with someone watching
✔ HTML/PDF documentation (e.g. with Sphinx)
✔ Published documentation
✔ A release on PyPI
For example:
https://github.com/scikit-learn-contrib/project-template