Ruling dependencies in big
projects
Mykola Palamarchuk
Upwork
lividgreen@gmail.com
https://github.com/lividgreen
twitter: @lividgreen
1.
Growing projects
Strategy of successful owning
Growing projects
“
“Divide and conquer” is gaining
and maintaining power by
breaking up larger concentrations
of power into pieces that
individually have less power than
the one implementing the strategy.
Growing projects: divide and conquer concept
In politics and sociology, divide and conquer
concept refers to a strategy that breaks up
existing power structures and prevents
smaller power groups from linking up.
This concept is either useful in project
management.
➔ divide per persons
➔ divide per teams
Growing projects: dependencies
Project parts have
dependencies on
each other.
How should we rule
them all to prevent
our parts from
collapsing into
monolithic one?
“
Dependency hell is a colloquial
term for the frustration of some
software users who have installed
software packages which have
dependencies on specific versions
of other software packages.
Wikipedia
2.
Static PHP code
dependencies
Reaching a successful compilation
of all the project code parts
PHP code dependencies: retrospective
1. svn externals, git modules etc.
2. pear
3. custom solutions (like Symfony1 plugins)
4. composer
Composer
helps us to manage
dependencies of our
PHP code
Composer
◎ Written in PHP, portable
◎ A lot of features, configurable
◎ Based on semver.org specification
Semver
MAJOR.MINOR.PATCH
◎ Declare public API as precise as possible
○ precise and unambiguous
○ concise
○ complete
◎ Use only features from public API of other
libraries
○ public methods
○ documentation!
Semver and common mistakes
Dependencies of your dependencies are
not your own dependencies (may
disappear!)
Add all dependencies you use to your
composer.json
Breaking changes in minor/patch release
Release new patch with rollback
changes
Composer and dependency hell
◎ Composer doesn’t allow different versions
of packages installed together
◎ Composer resolves your dependencies very
well (with clear error messages)
Composer depends on few Symfony
Components itself.
Composer and dependency hell: solutions?
◎ Avoid dependencies?
We must explore the nature of code
dependencies before making a conclusion
Composer and Big Projects: HOWTO
★ Profess semver.org everywhere
★ Use Composer aliases
○ test your contributions to open-source libraries in
integration
○ test feature branches of your libraries in
integration
★ Use Satis/Toran Proxy
3.
Dynamic
dependencies
Managing your runtime
components
Components
Framework
Common
functionality to run
your application
parts
Module
(plugin/bundle/...)
Named and
separated part of
your application
functionality/resourc
es.
Service
Single named
instance (object,
function, ...) that can
do something useful.
Components: Examples
Framework Modularity Services
Symfony 2 Bundles +
Yii 2 Extensions components
Laravel Packages +
Zend Framework 2 Modules +
Components: Problems
◎ Decomposition to modules
◎ Dependencies between modules
◎ Configuration
OOP
Can we use it here?
Module as object
Module is an object!
◎ bundle initialization = bundle constructor
◎ configuration = constructor parameters
◎ provided services = getter methods
◎ dependencies = dependencies
○ services may be passed to “constructor”
○ setter injection (good or evil?)
○ global services (evil?)
S.O.L.I.D.
Dependency Inversion Principle
Modules should not depend on each other
explicitly (in code). Use common interfaces to
connect modules.
Dependency inversion
Module 1 Module 2Service Constructor
Interface
DI
Module, DIP and code
◎ Separate repository for interface
psr-3 (logging), psr-7 (http messaging)
◎ Separate repository for provider module
○ depends on interface repository
◎ Separate repository for consumer module
○ depends on interface repository
○ no code dependencies between modules
Remember composer dependency hell
problem? Here is the solution!
Module configuration
◎ Semantic configuration API
○ configuration is a part of module API
○ no global dependencies in module
◎ Configure them all in your application
configuration
○ pass parameters to bundles
○ resolve dependencies
Example: Symfony
# app/config/config.yml
services:
sms_sender:
class: HelloSmsSender
arguments:
- "localhost:1234" # gateway
contacts:
class: HelloContactManager
arguments:
- @sms_sender
- "/var/contacts/db.txt"
<?php
namespace HelloContact;
use HelloSmsSender;
class Manager {
//...
public function __construct(Sender $smsSender, $dbPath) {
$this->smsSender = $smsSender;
$this->dbPath = $dbPath;
}
// ...
}
Example: Symfony (2)
<?php
namespace HelloSms;
interface SenderInterface {
public function send($number, $message);
}
Let’s create interface in separate code
repository
Example: Symfony (3)
<?php
namespace HelloBundleContact;
use HelloSmsSenderInterface;
class Manager {
//...
public function __construct(SenderInterface $smsSender, $dbPath) {
$this->smsSender = $smsSender;
$this->dbPath = $dbPath;
}
// ...
}
<?php
namespace SmsBundleSms;
use HelloSmsSenderInterface;
class Sender implements SenderInterface {
//...
}
move class Manager to
HelloBundle
move class Sender to
SmsBundle
Example: Symfony (4)
# app/config/config.yml
sms: # SmsBundle configuration
gateway: "localhost:1234"
hello: # HelloBundle configuration
sms_sender_id: "sms_sender"
file: "/var/contacts/db.txt"
# SmsBundleResourcesconfigconfig.yml
services:
sms_sender:
class: SmsBundleSmsSender
arguments:
- %sms.gateway%
# HelloBundleResourcesconfigconfig.yml
services:
contacts:
class: HelloBundleContactManager
arguments:
- @hello.sms_sender
- %hello.file_path%
◎ main configuration doesn’t depend on
bundles specific
◎ main configuration is clear
◎ raw bundle dependencies are eliminated
Some work have to be done in bundles
configuration handlers (so called “extensions”
in Symfony)
Example: Laravel
<?php
namespace MyCompanySms;
use IlluminateSupportServiceProvider;
class SmsServiceProvider extends ServiceProvider
{
public function register()
{
$this->app['sms.sender'] = $this->app->share(function($app) {
$gateway = $app['config']['mycompany/sms::gateway'];
return new Sender($gateway);
});
}
}
Example: Laravel (2)
<?php
namespace MyCompanyContact;
use IlluminateSupportServiceProvider;
class ContactsServiceProvider extends ServiceProvider
{
public function register()
{
$this->app['contacts'] = $this->app->share(function($app) {
$senderId = $app['config']['mycompany/contact::sms_sender_id'];
$file = $app['config']['mycompany/contact::file'];
return new Manager($app[$senderId], $file);
});
}
}
Is there any specification for component models?
The OSGi specification describes a modular
system and a service platform for the Java
programming language that implements a
complete and dynamic component model.
This one is for Java, but you should read it to
improve your understanding.
Eclipse IDE is built over OSGi.
Advantage of OSGi: dynamic module loading.
4.
Microservices
Separation of responsibilities
Microservice architecture
Balancer
Application 1
http://myapp.com/personal
Application 2
http://myapp.com/enterprise
Microservice dependencies
Rules are the same:
◎ depend on API instead of service itself
◎ use semver.org
○ Apache Thrift for data transfer and versioning
○ Docs
◎ service discovery
○ Consul
Microservice configuration
Problem: change configuration
simultaneously for every service
Solution: use Consul (or similar solution)
Consul KV - key-value storage with REST API
Consul-Template - agent that regenerates
config files (may run “cache clear”)
5.
The end?
Uncovered problems
Advanced topics
◎ Affecting services lifecycle
◎ Dependencies of listeners/middleware
◎ Configuration origins and environments
◎ Configuration policies
◎ Cross-language dependencies (php + js)
◎ Module hierarchy
◎ Lazy service initialization (Symfony
session)
◎ Dependencies in testing
◎ ...
Thanks!
Any questions?
Credits
Special thanks to all the people who made and released
these awesome resources for free:
◎ Presentation template by SlidesCarnival

Николай Паламарчук "Управление зависимостями в больших проектах"

  • 1.
    Ruling dependencies inbig projects Mykola Palamarchuk Upwork lividgreen@gmail.com https://github.com/lividgreen twitter: @lividgreen
  • 2.
  • 3.
  • 4.
    “ “Divide and conquer”is gaining and maintaining power by breaking up larger concentrations of power into pieces that individually have less power than the one implementing the strategy.
  • 5.
    Growing projects: divideand conquer concept In politics and sociology, divide and conquer concept refers to a strategy that breaks up existing power structures and prevents smaller power groups from linking up. This concept is either useful in project management. ➔ divide per persons ➔ divide per teams
  • 6.
    Growing projects: dependencies Projectparts have dependencies on each other. How should we rule them all to prevent our parts from collapsing into monolithic one?
  • 7.
    “ Dependency hell isa colloquial term for the frustration of some software users who have installed software packages which have dependencies on specific versions of other software packages. Wikipedia
  • 8.
    2. Static PHP code dependencies Reachinga successful compilation of all the project code parts
  • 9.
    PHP code dependencies:retrospective 1. svn externals, git modules etc. 2. pear 3. custom solutions (like Symfony1 plugins) 4. composer
  • 10.
    Composer helps us tomanage dependencies of our PHP code
  • 11.
    Composer ◎ Written inPHP, portable ◎ A lot of features, configurable ◎ Based on semver.org specification
  • 12.
    Semver MAJOR.MINOR.PATCH ◎ Declare publicAPI as precise as possible ○ precise and unambiguous ○ concise ○ complete ◎ Use only features from public API of other libraries ○ public methods ○ documentation!
  • 13.
    Semver and commonmistakes Dependencies of your dependencies are not your own dependencies (may disappear!) Add all dependencies you use to your composer.json Breaking changes in minor/patch release Release new patch with rollback changes
  • 14.
    Composer and dependencyhell ◎ Composer doesn’t allow different versions of packages installed together ◎ Composer resolves your dependencies very well (with clear error messages) Composer depends on few Symfony Components itself.
  • 15.
    Composer and dependencyhell: solutions? ◎ Avoid dependencies? We must explore the nature of code dependencies before making a conclusion
  • 16.
    Composer and BigProjects: HOWTO ★ Profess semver.org everywhere ★ Use Composer aliases ○ test your contributions to open-source libraries in integration ○ test feature branches of your libraries in integration ★ Use Satis/Toran Proxy
  • 17.
  • 18.
    Components Framework Common functionality to run yourapplication parts Module (plugin/bundle/...) Named and separated part of your application functionality/resourc es. Service Single named instance (object, function, ...) that can do something useful.
  • 19.
    Components: Examples Framework ModularityServices Symfony 2 Bundles + Yii 2 Extensions components Laravel Packages + Zend Framework 2 Modules +
  • 20.
    Components: Problems ◎ Decompositionto modules ◎ Dependencies between modules ◎ Configuration
  • 21.
  • 22.
    Module as object Moduleis an object! ◎ bundle initialization = bundle constructor ◎ configuration = constructor parameters ◎ provided services = getter methods ◎ dependencies = dependencies ○ services may be passed to “constructor” ○ setter injection (good or evil?) ○ global services (evil?)
  • 23.
    S.O.L.I.D. Dependency Inversion Principle Modulesshould not depend on each other explicitly (in code). Use common interfaces to connect modules.
  • 24.
    Dependency inversion Module 1Module 2Service Constructor Interface DI
  • 25.
    Module, DIP andcode ◎ Separate repository for interface psr-3 (logging), psr-7 (http messaging) ◎ Separate repository for provider module ○ depends on interface repository ◎ Separate repository for consumer module ○ depends on interface repository ○ no code dependencies between modules Remember composer dependency hell problem? Here is the solution!
  • 26.
    Module configuration ◎ Semanticconfiguration API ○ configuration is a part of module API ○ no global dependencies in module ◎ Configure them all in your application configuration ○ pass parameters to bundles ○ resolve dependencies
  • 27.
    Example: Symfony # app/config/config.yml services: sms_sender: class:HelloSmsSender arguments: - "localhost:1234" # gateway contacts: class: HelloContactManager arguments: - @sms_sender - "/var/contacts/db.txt" <?php namespace HelloContact; use HelloSmsSender; class Manager { //... public function __construct(Sender $smsSender, $dbPath) { $this->smsSender = $smsSender; $this->dbPath = $dbPath; } // ... }
  • 28.
    Example: Symfony (2) <?php namespaceHelloSms; interface SenderInterface { public function send($number, $message); } Let’s create interface in separate code repository
  • 29.
    Example: Symfony (3) <?php namespaceHelloBundleContact; use HelloSmsSenderInterface; class Manager { //... public function __construct(SenderInterface $smsSender, $dbPath) { $this->smsSender = $smsSender; $this->dbPath = $dbPath; } // ... } <?php namespace SmsBundleSms; use HelloSmsSenderInterface; class Sender implements SenderInterface { //... } move class Manager to HelloBundle move class Sender to SmsBundle
  • 30.
    Example: Symfony (4) #app/config/config.yml sms: # SmsBundle configuration gateway: "localhost:1234" hello: # HelloBundle configuration sms_sender_id: "sms_sender" file: "/var/contacts/db.txt" # SmsBundleResourcesconfigconfig.yml services: sms_sender: class: SmsBundleSmsSender arguments: - %sms.gateway% # HelloBundleResourcesconfigconfig.yml services: contacts: class: HelloBundleContactManager arguments: - @hello.sms_sender - %hello.file_path% ◎ main configuration doesn’t depend on bundles specific ◎ main configuration is clear ◎ raw bundle dependencies are eliminated Some work have to be done in bundles configuration handlers (so called “extensions” in Symfony)
  • 31.
    Example: Laravel <?php namespace MyCompanySms; useIlluminateSupportServiceProvider; class SmsServiceProvider extends ServiceProvider { public function register() { $this->app['sms.sender'] = $this->app->share(function($app) { $gateway = $app['config']['mycompany/sms::gateway']; return new Sender($gateway); }); } }
  • 32.
    Example: Laravel (2) <?php namespaceMyCompanyContact; use IlluminateSupportServiceProvider; class ContactsServiceProvider extends ServiceProvider { public function register() { $this->app['contacts'] = $this->app->share(function($app) { $senderId = $app['config']['mycompany/contact::sms_sender_id']; $file = $app['config']['mycompany/contact::file']; return new Manager($app[$senderId], $file); }); } }
  • 33.
    Is there anyspecification for component models? The OSGi specification describes a modular system and a service platform for the Java programming language that implements a complete and dynamic component model. This one is for Java, but you should read it to improve your understanding. Eclipse IDE is built over OSGi. Advantage of OSGi: dynamic module loading.
  • 34.
  • 35.
  • 36.
    Microservice dependencies Rules arethe same: ◎ depend on API instead of service itself ◎ use semver.org ○ Apache Thrift for data transfer and versioning ○ Docs ◎ service discovery ○ Consul
  • 37.
    Microservice configuration Problem: changeconfiguration simultaneously for every service Solution: use Consul (or similar solution) Consul KV - key-value storage with REST API Consul-Template - agent that regenerates config files (may run “cache clear”)
  • 38.
  • 39.
    Advanced topics ◎ Affectingservices lifecycle ◎ Dependencies of listeners/middleware ◎ Configuration origins and environments ◎ Configuration policies ◎ Cross-language dependencies (php + js) ◎ Module hierarchy ◎ Lazy service initialization (Symfony session) ◎ Dependencies in testing ◎ ...
  • 40.
  • 41.
    Credits Special thanks toall the people who made and released these awesome resources for free: ◎ Presentation template by SlidesCarnival