Technical Lag in npm and
Docker Ecosystems
Ahmed Zerouali
SoHeal Workshop, co-located with ICSE
Gothenburg, Sweden - May 27, 2018

/background

/background
Not updateUpdate “If I go there will be trouble,
And if I stay it will be double,
So come on and let me know:
Should I Stay Or Should I Go? ”
The Clash

/background
Technical lag*: the increasing difference between
deployed software packages and the ideal available
upstream packages
Measurement: version updates, bugs, vulnerabilities,
line of code, commits, etc.
(*) Gonzalez-Barahona, et al. "Technical Lag in Software Compilations: Measuring How Outdated a Software Deployment Is."
IFIP International Conference on Open Source Systems. Springer, Cham, 2017.
Gold standard: stability, security, functionality, etc.

/background
Example: different kinds of “gold standards” for Debian
Gold standard Scenario Candidate
Stability Isolated system, stable
functionality
Debian Stable
Functionality Cloud application Latest upstream
Security Reused containers Stable upstream

/background
Credits: https://exploring-data.com/vis/npm-packages-dependencies/

/Aim & case studies
- npm packages : the whole registry
- Docker containers based on Debian: 7,380
Goal: Analyze technical lag of software ecosystems.

/method
/technical lag
- Measurement = version updates, time
- version lag : version updates difference
- time lag: time difference
- Gold standard = being up to date.

/method
/technical lag
1.0.1 1.1.0 2.0.01.2.0 2.0.1
Dependency: D
npm package
version
Technical lag
- time lag = date(latest) - date(used)
- version lag = (∆Major, ∆Minor, ∆Patch)

/method
/technical lag
1.0.1 1.1.0 2.0.01.2.0 2.0.1
Dependency: D
npm package
version
Technical lag
- time lag (D)= date(2.1.0) - date(1.1.0)

/method
/technical lag
1.0.1 1.1.0 2.0.01.2.0 2.0.1
Dependency: D
npm package
version
Technical lag
1 minor
- time lag (D)= date(2.1.0) - date(1.1.0)

/method
/technical lag
1.0.1 1.1.0 2.0.01.2.0 2.0.1
Dependency: D
npm package
version
Technical lag
- time lag (D)= date(2.1.0) - date(1.1.0)
1 minor
1 major

/method
/technical lag
1.0.1 1.1.0 2.0.01.2.0 2.0.1
Dependency: D
npm package
version
Technical lag
1 minor
1 major 1 patch
- time lag (D)= date(2.1.0) - date(1.1.0)
- version lag (D) = (1,1,1)

/method
/technical lag
1.0.1 1.2.0 2.0.1
3.6.0 4.1.04.0.0 5.0.0
2.0.0 2.1.0
npm package: P
dependency: D
^1.0.0
*
^1.0.0 ^2.0.0
^1.0.0 = [ 1.0.0, 2.0.0 [
allowed

/method
/semantic versioning
Other: *, ==1.2.3, >1.2.3, <1.2.3, 1.2.x, 1.x.x

/method
/technical lag
1.0.1 1.2.0 2.0.1
3.6.0 4.1.04.0.0 5.0.0
2.0.0 2.1.0
npm package: P
dependency: D
^1.0.0
*
^1.0.0 ^2.0.0
^1.0.0 = [ 1.0.0, 2.0.0 [
allowed

/method
/technical lag
1.0.1 1.2.0 2.0.1
3.6.0 4.1.04.0.0 5.0.0
2.0.0 2.1.0
npm package: P
dependency: D
^1.0.0
Technical lag
*
^1.0.0 ^2.0.0
allowed
^1.0.0 = [ 1.0.0, 2.0.0 [

/method
/technical lag
1.0.1 1.2.0 2.0.1
3.6.0 4.1.04.0.0 5.0.0
2.0.0 2.1.0
npm package: P
dependency: D
^1.0.0
Technical lag = 0
*
^1.0.0 ^2.0.0
allowed
^1.0.0 = [ 1.0.0, 2.0.0 [

/results
/npm packages
For direct dependencies
npm package: P
Dependencies
Direct

/results
/npm packages
- Time las is increasing over time.
- More than 500K packages with about 4M package releases and 46M
dependencies.
For direct dependencies

/results
/npm packages
- Development dependencies have slightly more version lag than runtime dependencies
For direct dependencies

/results
/npm packages
- Caret (^) usage is increasing over time.
- Caret introduction coincides with Major version lag increase.
For direct dependencies

/results
/npm external applications
- Technical lag is higher in external applications.
- more than 600k repositories (6.2M dependencies)
For direct dependencies

/results
/npm external applications
For direct dependencies
- The usage of strict constraint is much higher in external applications

/results
/npm packages
npm package: P
Dependencies
For transitive dependencies

/results
/npm packages
For transitive dependencies
Time Lag Version lag

/results
/Docker containers

/method
/technical lag in Docker
1.0.1 1.1.0 2.0.01.2.1 2.1.0
Installed package: D
Technical lag
technical lag (D) =
∆Versions
∆Vulnerabilities
∆Bugs
Ideal Version
deployed
container
Included
Package
version

/results
/Docker
The majority of packages in Debian containers is up-to-date.

/results
/Docker
- Outdated Debian packages in Docker containers induce a median of version lag of 1
version.

/results
/Docker

/conclusion
- Technical lag can be a measurement of software health.
- TL can be a proxy for the effort needed to deploy the
ideal version.

Thank you