How to increase the technical health of your software?

1. Prof. Dr. Tom Mens Software Engineering Lab How to increase the TECHNICAL HEALTH of your software? tom.mens@umons.ac.be @tom_mens Business & Technology Club – Infopole Cluster TIC – 19 February 2019

2. SECO-ASSIST "Excellence of Science” Research Project 2018-2021 seco-assist.github.io@seco-assist

3. What is technical health of software? Internal point of view • Focus on internal software quality characteristics • Measure technical debt External point of view • Focus on dependencies to external software components • Measure technical lag

4. Internal software health Technical view Increase technical wealth by reducing your technical debt “a concept in programming that reflects the extra development work that arises when code that is easy to implement in the short run is used instead of applying the best overall solution” (Ward Cunningham, 1992) http://legacycoderocks.libsyn.com/technical-wealth-with-declan-wheelan

5. Internal software health Technical view Reduce your technical debt • By detecting internal quality problems and “bad smells” • Duplicated code • Obsolete code • Poorly structured code • Inadequate or incomplete tests • Performance problems • Potential bugs and security vulnerabilities • And improving these problems using manual or automated refactoring and restructuring

6. Implementation of SQALE model in SonarQube

7. Internal software health Social view Social debt “Unforeseen project cost connected to sub-optimal organizational-social structures”

8. Internal software health Social view Reduce social debt by removing community smells • Organisational silo High decoupling and lack of communication between tasks • Black cloud Lack of people able to bridge the knowledge and experience gap • Prima-donnas Seemingly condescending and egotistical behaviour, unreceptiveness to collaboration • Organisational skirmish Misalignment of organisational culture • …

9. What is technical health of software? Internal point of view • Focus on internal software quality characteristics • Measure technical debt External point of view • Focus on dependencies to external software components • Measure technical lag

10. External software health Technical view • Dependency hell • Unmaintained or outdated libraries • Backward incompatibilities • Incompatible software licenses • Issues in dependencies may affect your own software • Bugs • Security vulnerabilities • Missing new features • … • Technical lag https://chaoss.community

11. External Software Health Technical Lag How outdated is your current software deployment (collection of software components) w.r.t. the “ideal” situation ? where “ideal” = “most recent”; “most secure”; ”most stable”; “most compatible”; … https://chaoss.community

12. External Software Health Technical Lag References • “A formal framework for measuring technical lag in component repositories – and its application to npm.” A. Zerouali, T. Mens, J. Gonzalez-Barahona, A. Decan, E. Constantinou, G. Robles. Wiley Journal on Software Evolution and Process, February 2019 • “On the evolution of technical lag in the npm package dependency network.” A. Decan, T. Mens, E. Constantinou. IEEE Int’l Conf. Software Maintenance and Evolution, 2018 • “Technical lag in software compilations: Measuring how outdated a software deployment is.” J. Gonzalez-Barahona, P. Sherwood, G. Robles, D. Izquierdo. IFIP Int’l Conf. Open Source Systems, 2017

13. Guidelines based on an analysis of the JavaScript packaging ecosystem

14. Guideline 1: Inspect your potential dependencies • Is the target software component well-documented? • Is it well-tested? • Is its code well-written? (Cf. technical debt) • Does the component have performance problems? • Is its community active and responsive? (Cf. social debt) • Is the component popular? • Are there known bugs or security issues? • Is the software licence compatible?

15. Example: leftpad

16. Example: micropackages

17. Example: micropackages

18. Example: transitive dependencies https://npm.anvaka.com/

21. Guideline 2: Avoid “unhealthy” dependency tree • Strive for few dependencies in your software • Avoid transitive dependencies: It just takes one transitive component to break or compromise your software! • Avoid depending on micro-packages • Avoid depending on unmaintained/obsolete components

22. Law of increasing complexity

23. Law of increasing complexity https://npm.anvaka.com/

28. Guideline 3: Monitor your dependencies • Law of increasing complexity: • Number of transitive dependencies tends to grow over time • Law of declining quality: • Quality of dependent component may decrease over time • To upgrade or not to upgrade? • Upgrading benefits from bug fixes, security fixes and new features • But may introduce breaking changes

29. Guideline 3: Monitor your dependencies Use Continuous Integration/Deployement and automatic dependency monitoring tools

30. Example: breaking changes

31. Guideline 4: Adhere to semantic versioning and use dependency constraints

32. Guideline 4: Adhere to semantic versioning and use dependency constraints • From provider side: Informs your dependents about which releases are backwards incompatible • From consumer side: Allows to decide and control when to upgrade to newer releases of dependencies

33. Security vulnerabilities security exploit in 2017 “attackers entered its system in mid-May through a web-application vulnerability that had a patch available in March. In other words, the credit-reporting giant had more than two months to take precautions that would have defended the personal data of 143 million people from being exposed. It didn’t.” Wired Magazine, “Equifax Has No Excuse”, September 2017

34. Security vulnerabilities HeartBleed bug • Vulnerable code introduced in 2012 by simple programming mistake. • Allowed anyone on the Internet to read the memory of the systems protected by OpenSSL • Vulnerability discovered and traced in April 2014 • 0.5M servers certified by trusted authorities were believed to be a affected

35. “1 out of 3 dependents never update their dependency to a vulnerable package” A. Decan et al. “On the impact of security vulnerabilities in the npm package dependency network”, MSR 2018. Improper or too restrictive use of dependency constraints Dependent package is no longer actively maintained Maintainers are unaware of the vulnerability or the fix Fixed package version is incompatible "37% of websites include a JavaScript library with a known open source vulnerability.” T. Lauinger et al. "Thou Shalt Not Depend on Me: Analysing the Use of Outdated JavaScript Libraries on the Web", NDSS 2017.

36. Guideline 5: Secure your software • Use security monitoring tools • To discover vulnerabilities faster • To update vulnerable dependencies • Do not depend on unmaintained packages • Their vulnerabilities may take a long time to get fixed

37. Guideline 5: Secure your software Use security monitoring tools

38. Guideline 6: Help your consumers • Inform dependents about • incompatible upgrades • planned updates • deprecated features • Help your dependents to upgrade more easily • Provide (automated) migration guidelines • Provide alpha/beta releases • Test your changes on dependents before releasing updates • Backport security fixes to earlier vulnerable releases