1. On the Impact of Security Vulnerabilities in the
npm and RubyGems Dependency Networks
Ahmed Zerouali, Tom Mens, Alexandre Decan, Coen De Roover
The International Conference on Software Maintenance and Evolution Limassol,
Cyprus - October 7th, 2022
Published in Empirical Software Engineering Volume 27, Number 107 (2022).
https://doi.org/10.1007/s10664-022-10154-1

2. Background
Credits: libraries.io 2

3. Motivation
3
Heartbleed

4. Motivation
4
Top 10 Web Application Security Risks
Credits: OWASP

5. 5
Goal
Understand how vulnerabilities spread
in software dependency trees

6. Dependency
network
6
Eventually

7. Case studies
7

8. Example
dependency constraints
8
Semantic Versioning
package.json

9. Example: tilde ~
9
~1.2.0
After some time>
{1.0.0, 1.1.0, 1.2.0, 1.2.1}
1.2.1
~1.2.0
{1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.3.0}
1.2.1
T1
T2

10. Example: caret ^
10
^1.2.0
After some time>
{1.0.0, 1.1.0, 1.2.0, 1.2.1}
1.2.1
^1.2.0
{1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.3.0}
1.3.0
T1
T2

11. 11
1.2.1
After some time >
1.3.0
T1 T2
Example
^1.2.0
^1.2.0

12. 12
Case studies
Data extraction date: 12 January 2020
< Packages >
748k 94k
10k
Identified transitive dependencies for packages and GitHub projects
13k

13. Case studies: vulnerability reports
13
2,188 vulnerability reports 686 vulnerability reports

14. RQ0: How prevalent are disclosed vulnerabilities in npm and RubyGems
packages?
14
Evolution number of vulnerabilities affecting npm and RubyGems packages, grouped by
severity.
- The number of vulnerabilities is increasing exponentially for npm and
linearly for RubyGems.
- The proportion of high and critical vulnerabilities seems to increase
over time.

15. RQ1 : How much time elapses until a vulnerability is disclosed?
15
Cumulative proportion of disclosed vulnerabilities since the first affected package release
- In npm, critical vulnerabilities are disclosed faster.
- Vulnerabilities in npm are disclosed faster than in RubyGems.
- In npm, Malicious Package vulnerabilities are disclosed faster than other vulnerability types.

16. RQ2 : For how long do packages remain affected by disclosed vulnerabilities?
16
- Disclosed vulnerabilities in npm take a considerably shorter time to fix since the first
affected release.
- For npm, 17.8% of the fixed vulnerabilities needed more than 90 days after
their disclosure to be fixed, while this proportion is 10% for RubyGems
- Half of all disclosed vulnerabilities take more than 4 years to fix since their introduction,
compared to 7 years for disclosed RubyGems vulnerabilities.

17. RQ2 : For how long do packages remain affected by disclosed vulnerabilities?
17
Proportion of fixed vulnerabilities per severity, grouped by the type of the first
unaffected release.
- 65% of all disclosed vulnerabilities are fixed in patch releases.
- The severity of a vulnerability does not seem to have an impact on the first
release type in which the vulnerability is fixed.

18. RQ3 : To what extent are dependents exposed to their vulnerable dependencies?
18
One vulnerable package can expose a median of:
- 11 direct dependent packages (npm)
- 12 direct dependent packages (RubyGems)
In total, one single vulnerable package is responsible for
exposing:
- a median of 21 and a maximum of 213,851 (67.8%)
npm packages;
- a median of 19 and a maximum of 22,233 (60.2%)
RubyGems packages, respectively.

19. RQ3 : To what extent are dependents exposed to their vulnerable dependencies?
19
Monthly evolution of the distribution of the number of vulnerabilities coming from transitive
dependencies of all studied packages.
- Older packages are exposed to more vulnerabilities coming from their dependencies than
recent ones.
- The introduction of “caret ^” constraint in 2014 led to a decrease in the number of
vulnerabilities in npm supply chain.

20. RQ4 : How are vulnerabilities spread in the dependency tree?
20
- The number of dependency vulnerabilities for packages decreases at deeper levels of the
dependency tree.
- Vulnerable dependencies continue to be found at the deepest levels (24 for npm and 16
for RubyGems).
Distribution of the number of vulnerabilities found in package dependencies, grouped by
dependency tree level

21. RQ5 : To what extent are dependents exposed to their vulnerable dependencies
at their release time?
21
- More than 50% of the package releases that are exposed to vulnerabilities via their
dependencies at the observation date, were not exposed to any disclosed vulnerabilities
when they were first created.
- At the time of their last commit, GitHub projects that make use of RubyGems packages
had proportionally more vulnerable direct dependencies than projects with npm
dependencies (33.9% compared to 22.1%).

22. Conclusion
22
Zerouali, A., Mens, T., Decan, A. and De Roover, C.
On the impact of security vulnerabilities in the npm and RubyGems dependency networks.
Empirical Software Engineering Volume 27, Number 107 (2022).
- The number of vulnerabilities is increasing over time, alongside the number of vulnerable
packages and their exposed dependents.
- More effort is needed. All parties can help.
https://doi.org/10.1007/s10664-022-10154-1

23. RQ0: How prevalent are disclosed vulnerabilities in npm and RubyGems
packages?
23
Top 10 vulnerability types

24. RQ3 : To what extent are dependents exposed to their vulnerable dependencies?
24
- 8.1% of the direct dependencies of GitHub projects on npm are vulnerable, while this is 10.9%
for RubyGems.
- npm-dependent projects have more highly vulnerable direct dependencies than
RubyGems-dependent projects.
- Only 3.2% of the indirect npm dependencies of GitHub projects are vulnerable, while this is
more than three times higher (10.5%) for RubyGems.