Download as PDF, PPTX
Uploaded byRoberto Pérez Alcolea
Dependency Management in a Complex World (JConf Chicago 2022)
The document discusses the challenges of dependency management in software development, highlighting issues like compatibility, build hygiene, and security vulnerabilities. It emphasizes the importance of frequent builds and releases, while proposing tools and strategies for better dependency resolution and observability within the Netflix ecosystem. The text underscores that while dependency hell is a common issue, there are solutions available to alleviate related difficulties.
More Related Content
Similar to Dependency Management in a Complex World (JConf Chicago 2022)
![[DPE Summit] How Improving the Testing Experience Goes Beyond Quality: A Deve...](https://cdn.slidesharecdn.com/ss_thumbnails/dpesummitseptember20-212023-howimprovingthetestingexperiencegoesbeyondqualityadeveloperproductivityp-230924000523-453e8af8-thumbnail.jpg?width=640&height=640&fit=bounds)
Dependency Management in a Complex World (JConf Chicago 2022)
- 1. Dependency Management in a ComplexWorld September 28, 2022
- 2. Roberto Perez Alcolea SeniorSoftware Engineer @ Net fl ix JVM Ecosystem Team rperezalcolea@net fl ix.com @rpalcolea Dependency Management in a Complex World
- 3. Life of a developer DependencyManagement in a Complex World
- 4. Dependency Management ina Complex World Tuesday Afternoon
- 5. Dependency Management ina Complex World Wednesday Morning
- 6. Dependency Management ina Complex World
- 7. Dependency Management ina Complex World Are we in dependency hell?
- 8. Dependency Management ina Complex World Most likely, yes! 🔥😱 🔥😱 🔥😱
- 9. Dependency Management ina Complex World This is just one project… what if…
- 10. Dependency Management ina Complex World Your dependency graph looks like this
- 11. Dependency Management ina Complex World — Multi/Many repositories setup (~3k) — Binary integration (JARs) — Microservices with fat clients — Hundreds/Thousands of Engineers — Thousands of builds per day — Thousands of artifacts generated per day
- 12. Dependency Management ina Complex World
- 13. Dependency Management ina Complex World How to keep up with all these changes? Build often, release often!
- 14. Dependency Management ina Complex World Yes, but this comes with some costs 💸 💸
- 15. Common Problems in dependency Management (That wehave seen) Dependency Management in a Complex World
- 16. Compatibility issues — 2components depending on the same module but with different, incompatible, APIs — Multiple libraries providing the same feature (logger APIs) — Dealing with incompatible versions of a runtime (e.g: Scala 2.11 vs Scala 2.12) — Misaligned dependencies of a component (e.g: Jackson Databind 2.9.0 with Jackson Core 2.9.4) — Intentional and unintentional Breaking API changes in libraries — Java version Dependency Management in a Complex World
- 17. Build / Dependencygraph Hygiene — Build does not declares the dependencies it needs — Build declares dependencies that doesn’t need Dependency Management in a Complex World
- 18. Publishers lack visibility —Who uses my library? — Who will I break if I release a change? — Who is exercising/using a particular method I want to deprecate and/or replace? Dependency Management in a Complex World
- 19. Semver is notenough — Which version should I use? — Pinning versions is technical debt — In a dynamic environment where projects get changes constantly, moving them between majors can be challenging Dependency Management in a Complex World
- 20. Constant change — Howcan we migrate a project from dependency A to B? — Can it be done only by dependency management machinery rules? — Do we need to ship code changes also? — Can we do this on behalf of the users? Dependency Management in a Complex World
- 21. Security Vulnerabilities — Preventprojects from using a given version of a dependency — Reject a particular module from the dependency graph — Let users know that they might be in danger Dependency Management in a Complex World
- 22. Ownership When my projectis broken… — Who owns the module/library breaking my project? — How can I contact them? — Is there a support channel to reach out? — Can I open an incident directly with them? Dependency Management in a Complex World
- 23. Are there any solutions? DependencyManagement in a Complex World
- 24. Dependency Management ina Complex World Sure, there are 😃😃!
- 25. Dependency Management ina Complex World Dependency Resolution — Resolution Rules Plugin — Resolution Rules — Gradle Plugins with Resolution Strategies — Gradle Module Metadata
- 26. Dependency Management ina Complex World Build Hygiene — Nebula Lint plugin — Dependency Analysis Gradle Plugin
- 27. Dependency Management ina Complex World Who uses my library? Collection of services and UIs that enable artifact observability and the ability to effect change in the Netflix ecosystem. Astrid to the rescue ⛑!
- 28. Dependency Management ina Complex World Who uses my library?
- 29. Dependency Management ina Complex World Who uses my library?
- 30. Dependency Management ina Complex World Who uses my library?
- 31. Dependency Management ina Complex World But what about security vulnerabilities?
- 32. Dependency Management ina Complex World CVE + Astrid + Nebula Practical Approach to Automate the Discovery & Eradication of Open-Source Software Vulnerabilities Aladdin Almubayed | 2019 ▶
- 33. Dependency Management ina Complex World Astrid Campaign
- 34. Dependency Management ina Complex World And you said constant change…
- 35. Dependency Management ina Complex World Automated dependencies and lint updates
- 36. Dependency Management ina Complex World Automated dependencies and lint updates
- 37. Dependency Management ina Complex World Automated dependencies and lint updates
- 38. Dependency Management ina Complex World Managed Delivery
- 39. Dependency Management ina Complex World Paved Path vs non-Paved Path (log4j edition) ~minutes-hour vs ~hours-days
- 40. Dependency Management ina Complex World Paved Path vs non-Paved Path (log4j edition)
- 41. Dependency Management ina Complex World Who owns this library?
- 42. Dependency Management ina Complex World Software Ownership
- 43. Dependency Management ina Complex World Unfortunately, some of these things are not OSS Today. However there are OSS and Enterprise options…
- 44. Dependency Management ina Complex World Update dependencies Security Constant change Code usage Refactoring Comby Ownership
- 45. What’s next for us? DependencyManagement in a Complex World
- 46. Publisher feedback — Whena new version of a library is published, build all the consumers and verify if the change breaks them. • This requires proper verification steps on each project — Use Sourcegraph Code Intelligence capabilities to understand when a modified/removed code is being used by consumers or not Dependency Management in a Complex World
- 47. Dependency Management ina Complex World Distributed Refactoring
- 48. Dependency Management ina Complex World Library changesets with publications
- 49. Last thoughts… — Dependencymanagement is hard — Dependency hell is inevitable, let’s try to reduce the pain — Build often, release often. Avoid conflict resolution by limiting version skew — Producers and consumers play an important role — Moving projects from A to B is hard but there are options out there Dependency Management in a Complex World
- 50. Questions? Dependency Management ina Complex World
- 51.