Just for you: FREE 60-day trial to the world’s largest digital library.
The SlideShare family just got bigger. Enjoy access to millions of ebooks, audiobooks, magazines, and more from Scribd.
Cancel anytime.
- 1. Tom Mens Eleni Constantinou Alexandre Decan Ahmed Zerouali Software Engineering Lab tom.mens@umons.ac.be @tom_mens econst@gmail.com @eleni_const Empirically Analysing the Socio-Technical Health of Software Package Managers
- 2. SECO-ASSIST "Excellence of Science” Research Project 2018-2021 secoassist.github.io@secoassist
- 3. Tom Mens University of Mons Eleni Constantinou University of Mons Bram Adams Polytechnique Montreal Josianne Marsan Laval University Towards an interdisciplinary, socio-technical methodology and analysis of the health of software ecosystems secohealth.github.io @secohealth
- 4. SOCIO- TECHNICAL A Software Ecosystem is X
- 5. TECHNICAL HEALTH Ecosystem Health Issues • Outdated dependencies • Security vulnerabilities • Bugs • Dependency hell • Co-installability issues • Backward incompatibility • Abandonment of contributors • Lack of communication / interaction • Social conflicts • Insufficient diversity
- 6. Comparing the health of dependency networks of popular software package managers
- 7. Comparing the health of dependency networks of popular software package managers A Decan et al. (Feb. 2019) An Empirical Comparison of Dependency Network Evolution in Seven Software Packaging Ecosystems. Emp. Softw. Eng. 24(1), Springer A Decan, T Mens (May 2019) What do package dependencies tell us about semantic versioning? IEEE Trans. Softw. Eng. [In Press] A Zerouali et al. (Feb. 2019) A formal framework for measuring technical lag in component repositories – and its application to npm? J. Softw. Evolution and Process A Decan et al. (2018) On the impact of security vulnerabilities in the npm package dependency network. Int’l Conf. MSR E Constantinou et al. (2017) An empirical comparison of developer retention in the Rubygems and npm software ecosystems. Innovations in Systems and Softw. Eng. 13(2-3) E Constantinou, T Mens (2017) Socio-technical evolution of the Ruby ecosystem in GitHub. Int’l Conf. SANER
- 8. Libraries.io monitors 3,903,576 open source packages across 36 different package managers
- 9. Characterising the evolution of package dependency networks Decan & Mens (2019) An Empirical Comparison of Dependency Network Evolution in Seven Software Packaging Ecosystems. Empirical Software Engineering 830K packages – 5.8M package versions – 20.5M dependencies (April 2017)
- 10. Characterising the evolution of package dependency networks • Growth rate • Package update frequency • Connectedness of dependency network • Prevalence of transitive dependencies
- 11. Motivation Package updates may cause many maintainability issues or even failures in dependent packages. "Especially with respect to package dependencies, the risk of things breaking at some point due to the fact that a version of a dependency has changed without you knowing about it is immense. That actually cost us weeks and months in a couple of professional projects I was part of."
- 12. March 2016 Unexpected removal of left-pad caused > 2% of all packages to become uninstallable (> 5,400 packages) November 2010 Release 0.5.0 of i18n broke dependent package ActiveRecord that was transitively required by >5% of all packages Motivation
- 13. Example: leftpad
- 14. Package dependency networks grow over time
- 15. Dependency networks grow over time
- 16. Package changes are frequent Findings • #package updates grows over time • >50% of package releases are updated within 2 months • Required and young packages are updated more frequently Changeability index: Maximal value n such that there exist n packages having been updated at least n times during the last month.
- 17. Most packages depend on other packages Findings • 60% to 80% of all packages are connected • A stable minority (20%) of required packages collect over 80% of all reverse dependencies • # npm dependencies grows much faster Reusability index: Maximal value n such that there exist n required packages having at least n dependent packages.
- 18. Proportion of top-level packages by depth of dependency tree Dependency network complexity Most of the complexity is deeply hidden … … in the transitive dependencies Over 50% of top-level packages have deep dependency tree.
- 19. Package changes may have important transitive impact Evolution of 5-Impact Index Findings • Dependent packages have few direct but many transitive dependencies • Ratio of indirect over direct dependencies increases over time P-Impact Index : Number of packages that are transitively required by at least P% of all packages.
- 20. Avoid packages with outdated dependencies 1 out of 3 dependents never � update their dependency A Zerouali et al (Feb. 2019) A formal framework for measuring technical lag in component repositories – and its application to npm. Wiley Journal on Software Evolution and Process
- 21. Avoid packages with outdated dependencies https://chaoss.community Should package maintainers upgrade their dependencies to more recent versions? �Upgrades benefit from bug and security fixes � Upgrading allows to use new features �Upgrading requires effort � Upgrading may introduce breaking changes
- 22. Semantic Versioning • For package providers: Inform your dependents about which releases are backwards incompatible • For package consumers: Decide and control which newer dependency releases are permitted major minor patch 3 9 2 Breaking changes Backwards compatible changes Bug fixes
- 23. Technical Lag in npm https://chaoss.community Technical lag measures how outdated a package or dependency is w.r.t. the “ideal” situation where “ideal” = “most recent”; “most secure”; ”most stable”; “most compatible”; …
- 24. All Dependencies Outdated Dependencies Technical Lag in npm Outdatedness is related to the type of dependency constraint being used Strict constraints represent about 20% of all dependencies, but about 33% of all outdated dependencies
- 25. Suggestion: Rely on semantic versioning when defining dependency constraints A Decan, T Mens (May 2019) What do package dependencies tell us about semantic versioning? Transactions on Software Engineering, IEEE [In Press] Does npm behave similarly to other package managers (Cargo, Packagist and RubyGems)? January 2018 dataset from libraries.io:
- 26. Suggestion: Rely on semantic versioning when defining dependency constraints Different package managers interpret version constraints in different ways: More restrictive than semver More permissive than semver
- 27. Suggestion: Rely on semantic versioning when defining dependency constraints Proportion of dependency constraints (for package releases ≥1.0.0) that are semver-compliant, more permissive, or more restrictive
- 28. Suggestion: Rely on semantic versioning when defining dependency constraints • Cargo, npm and Packagist are mostly semver- compliant. • All considered ecosystems become more compliant over time. • More than 16% of the constraints in npm, Packagist and Rubygems are restrictive, preventing backward compatible upgrades from being automatically adopted.
- 29. Security vulnerabilities Vulnerable code introduced in 2012 • Allowed anyone on the Internet to read the memory of the systems protected by OpenSSL • Simple programming mistake Discovered and traced in April 2014 • 0.5M servers certified by trusted authorities were believed to be a affected
- 30. Security vulnerabilities Vulnerability introduction Vulnerability discovery Vulnerability publication Vulnerability fixed time
- 31. Security vulnerabilities in OSS
- 32. Security vulnerabilities in npm Package metadata from libraries.io on November 2017 610,097 packages Packages releases 4,202,099 Runtime package dependencies 20,240,402 399 vulnerabilities Affected packages 269 # releases of affected packages 14,931 # affected releases 6,752 Vulnerabilities from snyk.io
- 33. When are vulnerabilities discovered? Vulnerability introduction Vulnerability discovery Vulnerability publication Vulnerability fixed time
- 34. When are vulnerabilities discovered? >40% of all vulnerabilities are not discovered even 2 years after their introduction, regardless of their severity.
- 35. Tool support should strive to discover vulnerabilities sooner. When are vulnerabilities discovered?
- 36. When are vulnerabilities fixed? Vulnerability introduction Vulnerability discovery Vulnerability publication Vulnerability fixed time
- 37. When are vulnerabilities fixed? + Most vulnerabilities are quickly fixed after their discovery. - ~20% of vulnerabilities take more than 1 year to be fixed. Most vulnerabilities are fixed after the reported discovery date but before they become public.
- 38. When are vulnerabilities fixed? Vulnerabilities must be fixed early/before public announcement. Unmaintained vulnerable packages should be deprecated.
- 39. When are vulnerabilities fixed in dependent packages? Vulnerable packages # vulnerable packages 269 # releases of vulnerable packages 14,931 # vulnerable releases 6,752 # dependent packages 133,602 # dependent packages affected by the vulnerable packages 72,470
- 40. Unfixed dependent packages Improper or too restrictive use of dependency constraints Dependent package is no longer actively maintained Maintainers of dependent packages are not aware of the vulnerability or the fix Fixed version of the dependency contains incompatible changes
- 41. Security vulnerabilities in npm Use security monitoring tools Deprecate unmaintained obsolete packages Use better versioning and security policies
- 42. Socio-technical evolution r1.0.0 r1.1.0 r1.2.0 r1.3.0 r1.0.0 r1.0.1 r1.1.0 Project 1 Project 2 Project 5 Project 4 Project 3
- 43. Socio-technical evolution 144K packages 70K repositories 42K contributors E Constantinou, T Mens (Feb 2017) Socio-technical evolution of the Ruby ecosystem in GitHub. SANER 2017
- 44. Socio-technical evolution metrics 𝑇𝑒𝑎𝑚𝑅𝑒𝑛𝑒𝑤𝑎𝑙(𝑡) = |𝐽𝑜𝑖𝑛𝑒𝑟𝑠(𝑡)| | 𝑐 𝑖𝑠𝐶𝑜𝑛𝑡𝑟(𝑐, 𝑡)}| 𝑇𝑒𝑎𝑚𝐴𝑏𝑎𝑛𝑑𝑜𝑛𝑚𝑒𝑛𝑡(𝑡) = |𝐿𝑒𝑎𝑣𝑒𝑟𝑠(𝑡)| | 𝑐 𝑖𝑠𝐶𝑜𝑛𝑡𝑟(𝑐, 𝑡 − 1)}| 𝑃𝑟𝑜𝑗𝑒𝑐𝑡𝑅𝑒𝑛𝑒𝑤𝑎𝑙(𝑡) = |𝑁𝑒𝑤𝑃𝑟𝑜𝑗𝑒𝑐𝑡𝑠(𝑡)| | 𝑝 𝑖𝑠𝐴𝑐𝑡𝑖𝑣𝑒(𝑝, 𝑡)}| Project𝐴𝑏𝑎𝑛𝑑𝑜𝑛𝑚𝑒𝑛𝑡(𝑡) = |𝑂𝑏𝑠𝑜𝑙𝑒𝑡𝑒𝑃𝑟𝑜𝑗𝑒𝑐𝑡𝑠(𝑡)| | 𝑝 𝑖𝑠𝐴𝑐𝑡𝑖𝑣𝑒(𝑝,𝑡−1)}|
- 45. Social analysis 2008 2009 2010 2011 2012 2013 2014 2015 2016 0 0.2 0.4 0.6 0.8 1 TeamRenewal TeamAbandonment
- 46. Technical analysis 2008 2009 2010 2011 2012 2013 2014 2015 2016 0 0.2 0.4 0.6 0.8 1 ProjectRenewal ProjectAbandonment
- 47. Socio-technical impact 2008 2009 2010 2011 2012 2013 2014 2015 0 1 2 3 4 Specialization Specialization (Kullback-Liebler divergence) Project i importance for contributor Cj Project i importance in ecosystem
- 48. Abandoner characteristics How to determine the characteristics of contributors abandoning the ecosystem? Survival analysis Population: contributors in an ecosystem Event: leaving the ecosystem E Constantinou, T Mens (2017) An empirical comparison of developer retention in the Rubygems and npm software ecosystems. Innovations in Systems and Software Engineering 13(2-3), Springer
- 49. Abandoner characteristics ~ 144K packages ~ 32K contributors ~ 462K packages ~ 64K contributors Measurements on: • Technical activity • Social activity Intensity Frequency Duration of inactivity
- 50. Abandoner characteristics 0 50 100 150 200 0.00.20.40.60.81.0 npm Duration of commit activity (months) Survivalprobability Social inactivity Social activity Social abandoner 0 50 100 150 0.00.20.40.60.81.0 RubyGems Duration of commit activity (months) Survivalprobability Social inactivity Social activity Social abandoner 0 50 100 150 200 0.00.20.40.60.81.0 npm Duration of commit activity (months) Survivalprobability Very Short Short Long Very Long 0 50 100 150 0.00.20.40.60.81.0 RubyGems Duration of commit activity (months) Survivalprobability Very Short Short Long Very Long
- 51. Abandoner characteristics Developers tend to abandon the ecosystem sooner if they: do not communicate communicate or commit less intensively communicate or commit less frequently do not communicate or do not commit for a longer period of time
- 52. Activity prediction
- 53. Summary
Editor's Notes
Recommended
More Related Content
Related Books
Free with a 30 day trial from Scribd