On the Impact of Security Vulnerabilities in the npm and RubyGems Dependency Networks
1. On the Impact of Security Vulnerabilities in the
npm and RubyGems Dependency Networks
Ahmed Zerouali, Tom Mens, Alexandre Decan, Coen De Roover
The International Conference on Software Maintenance and Evolution Limassol,
Cyprus - October 7th, 2022
Published in Empirical Software Engineering Volume 27, Number 107 (2022).
14. RQ0: How prevalent are disclosed vulnerabilities in npm and RubyGems
Evolution number of vulnerabilities affecting npm and RubyGems packages, grouped by
- The number of vulnerabilities is increasing exponentially for npm and
linearly for RubyGems.
- The proportion of high and critical vulnerabilities seems to increase
15. RQ1 : How much time elapses until a vulnerability is disclosed?
Cumulative proportion of disclosed vulnerabilities since the ﬁrst affected package release
- In npm, critical vulnerabilities are disclosed faster.
- Vulnerabilities in npm are disclosed faster than in RubyGems.
- In npm, Malicious Package vulnerabilities are disclosed faster than other vulnerability types.
16. RQ2 : For how long do packages remain aﬀected by disclosed vulnerabilities?
- Disclosed vulnerabilities in npm take a considerably shorter time to ﬁx since the ﬁrst
- For npm, 17.8% of the ﬁxed vulnerabilities needed more than 90 days after
their disclosure to be ﬁxed, while this proportion is 10% for RubyGems
- Half of all disclosed vulnerabilities take more than 4 years to ﬁx since their introduction,
compared to 7 years for disclosed RubyGems vulnerabilities.
17. RQ2 : For how long do packages remain aﬀected by disclosed vulnerabilities?
Proportion of ﬁxed vulnerabilities per severity, grouped by the type of the ﬁrst
- 65% of all disclosed vulnerabilities are ﬁxed in patch releases.
- The severity of a vulnerability does not seem to have an impact on the ﬁrst
release type in which the vulnerability is ﬁxed.
18. RQ3 : To what extent are dependents exposed to their vulnerable dependencies?
One vulnerable package can expose a median of:
- 11 direct dependent packages (npm)
- 12 direct dependent packages (RubyGems)
In total, one single vulnerable package is responsible for
- a median of 21 and a maximum of 213,851 (67.8%)
- a median of 19 and a maximum of 22,233 (60.2%)
RubyGems packages, respectively.
19. RQ3 : To what extent are dependents exposed to their vulnerable dependencies?
Monthly evolution of the distribution of the number of vulnerabilities coming from transitive
dependencies of all studied packages.
- Older packages are exposed to more vulnerabilities coming from their dependencies than
- The introduction of “caret ^” constraint in 2014 led to a decrease in the number of
vulnerabilities in npm supply chain.
20. RQ4 : How are vulnerabilities spread in the dependency tree?
- The number of dependency vulnerabilities for packages decreases at deeper levels of the
- Vulnerable dependencies continue to be found at the deepest levels (24 for npm and 16
Distribution of the number of vulnerabilities found in package dependencies, grouped by
dependency tree level
21. RQ5 : To what extent are dependents exposed to their vulnerable dependencies
at their release time?
- More than 50% of the package releases that are exposed to vulnerabilities via their
dependencies at the observation date, were not exposed to any disclosed vulnerabilities
when they were ﬁrst created.
- At the time of their last commit, GitHub projects that make use of RubyGems packages
had proportionally more vulnerable direct dependencies than projects with npm
dependencies (33.9% compared to 22.1%).
Zerouali, A., Mens, T., Decan, A. and De Roover, C.
On the impact of security vulnerabilities in the npm and RubyGems dependency networks.
Empirical Software Engineering Volume 27, Number 107 (2022).
- The number of vulnerabilities is increasing over time, alongside the number of vulnerable
packages and their exposed dependents.
- More effort is needed. All parties can help.
23. RQ0: How prevalent are disclosed vulnerabilities in npm and RubyGems
Top 10 vulnerability types
24. RQ3 : To what extent are dependents exposed to their vulnerable dependencies?
- 8.1% of the direct dependencies of GitHub projects on npm are vulnerable, while this is 10.9%
- npm-dependent projects have more highly vulnerable direct dependencies than
- Only 3.2% of the indirect npm dependencies of GitHub projects are vulnerable, while this is
more than three times higher (10.5%) for RubyGems.