Continuous Integration and Automated
Deployments for Symfony-based projects
Symfony 2 under control
PHP Minsk User Group
INTRODUCTION
2
Max Romanovsky
maxromanovsky.com
max.romanovsky@gmail.com
Maxim_Romanovsky@epam.com
Twitter, LinkedIn, SensioLabs Connect,...
Continuous integration (CI) is the practice, in
software engineering, of merging all
developer working copies with a share...
• Improve quality of application
• Understand current situation on project
• Automate your checklists & verifications
• No...
Dependencies
• Composer
• Platform
packages
• Phar
• Pear
• Pecl
Build
• Phing
• phpqatools.org
Continuous
Integration
• J...
DEPENDENCIES
7
Manages 3rd party
dependencies
required at runtime*
• Bundles
• PHP libraries
• JS/CSS libraries
• *Phpunit for hphpa
8
Co...
Checklist
• stability
• lib-*, ext-*
• install vs update
• require
• version masks
• Keep composer.phar
under VCS
• Satis ...
Phar
• Only for CLI libraries used for development and build
• Composer, php-cs-fixer
Pear
• Build tools
• Phing, phpunit*...
What to update?
• Composer dependencies
• PHARs
• PEARs
• PECLs
• Platform packages
Keep in mind
• Update at the beginning...
BUILD
12
Ant
• From Java world
• XML declarative config
• Can run tasks in parallel
• No PHP-specific tasks
• Could be extended (Ja...
build-symfony-
ci.xml
• Default workflow
build.xml
• Overwrites for current
project
build.properties.dist
• Default config...
Target
• A set of tasks
• Could depend
on other targets
• phing
<target_name>
Task
• Action
executed as a
part of target
•...
build
• for dev
environment
• outputs to
the console
build-ci
• for CI
environment
• outputs to
XML files
package
• genera...
Special clean
prepare
build-tools-self-update
Composer-
specific
composer-validate
dependencies-install
dependencies-list-...
General phplint
php -l
phploc
pdepend
charts
phpmd
+ custom config
phpcs/phpcs-js
+ custom config (PSR-1&2, no
var_dump()....
General apidoc
PHPDoc
projectdoc
rst2html
phpunit
via exec
phpcb
PHP_CodeBrowser UI
php-cs-fixer
hphpa
HipHop static analy...
Symfony-
specific
copy-configs-ci parameters.yml for CI
cache:warmup app/cache is removed
during build
twig:lint for Bundl...
Symfony-
specific
doctrine:database:drop
doctrine:database:create
doctrine:migrations:migrate
doctrine:fixtures:load
doctr...
Package package-version version.txt
package-db-schema schema.sql
package-archive
Deployment deploy-maintenance- enable/dis...
CONTINUOUS
INTEGRATION
23
CruiseControl
• +PHP Under
Control
TeamCity
• Commercial
Travis CI
• SaaS
Scrutinizer CI
• SaaS
• PHP-oriented
Jenkins CI
...
• Open-source
• Enterprise
• Lot of plugins
• PHP Template from Sebastian
Bergmann
• IDE Integration
• RESTful API
25
Jenk...
• No comments git
• green & red signs for buildsgreenballs
• parameterized builds
parameterized-
trigger
• git tag select...
phing
• phpcs
• hphpacheckstyle
• phpcpddry
• php_dependjdepend
• phploc using CSV filesplot
• phpmdpmd
• aggregates info ...
Master
• Bleeding edge
• For completed features
Feature
branch
• Merge to master on completion
• After code review
• Devel...
• Builds specific branch (master)
• Features are verified after merge
build-<branchname>
• Parameterized
• Produces .tar.g...
• Produces .tar.gz artifact from latest revision of branch
• Deploys to QA
• Not recommended to have more than one such jo...
Deployed .tar.gz archive
• <tag>.<build#>
• v0.0.3.26
• <branchname>-<build#>-<revision>-SNAPSHOT
• master-12-7c041a90366c...
DEPLOYMENT
32
Capistrano
• from Ruby world
Capifony
• Based on
Capistrano
Shell script
• VCS update
• Rsync
Phar
• Don’t use WebPhar
• S...
Based on Capistrano
• Supports Capistrano plugins
Symfony 1&2 deployment
Multistage
• qa, staging, uat, production
Mainten...
Directories
• releases/
• current/ (symlink)
• shared/
Shared
• app/config/parameters.yml
• app/cache
• app/logs
• web/upl...
`-- /var/www/my-app.com
|-- current → /var/www/my-app.com/releases/20100512131539
|-- releases
| `-- 20100512131539
| `-- ...
All you need is package
• capistrano-deploy-strategy-archive
Github not available?
• Goodbye, vendors!
Decreases deploymen...
.tar.gz
Optimized
autoloader
Contains only
necessary
dirs & files
assets:install assetic:dump
version.txt &
db-
schema.sql...
DBDeploy Liquibase
Doctrine
Migrations
Doctrine
Fixtures
Database Migrations
39
User
• Triggers job on Jenkins CI
• Choose options (env, maintenance mode)
Jenkins
• Executes single phing target with add...
Enables Maintenance mode
• maintenance.html in webroot
Backups DB
Clears Doctrine cache
• Not for APC cache
Downloads prev...
Restarts services
• php-fpm or Apache
• Doctrine APC cache is cleared
Symfony-specific tasks
• copy app/config/parameters....
Semi-
automatical
• from
deployment
server
• usually CI
server
Enable
Maintenance
• manually via
phing
Manually backup
for...
Switch to
previous release
• Manually via
capifony CLI
• Recover DB
• Rollback
Doctrine
Migrations
• Restore DB
dump
• Cus...
Write rollback guide
• Step-by-step
• Verify it periodically
Enable monitoring on prod
• Zabbix
• Munin
• New Relic
• Pinb...
SUMMARY
46
• Currently not available online 
• Planning to release in 1-2 months on
GitHub
Where to download?
47
Deployment
•Cronjob / CLI script handling
•Web server (Apache, nginx)
config manipulations
•Ext- and lib- dependency
verif...
Questions?
Upcoming SlideShare
Loading in...5
×

Symfony Under Control by Maxim Romanovsky

4,134

Published on

В продолжение темы непрерывной интеграции, Макс расскажет о своем подходе организации непрерывной интеграции и деплоймента в Symfony проектах. Рассказ включает следующие темы:
- Управления зависимостями
- Процесс и инструменты для сборки
- Сервера непрерывной интеграции и в частности Jenkins, плагины к нему, jobs
- Процесс разработки в git
- Процесс выгрузки релиза
- Миграция БД
- Откат релиза

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,134
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
15
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Symfony Under Control by Maxim Romanovsky

  1. 1. Continuous Integration and Automated Deployments for Symfony-based projects Symfony 2 under control PHP Minsk User Group
  2. 2. INTRODUCTION 2
  3. 3. Max Romanovsky maxromanovsky.com max.romanovsky@gmail.com Maxim_Romanovsky@epam.com Twitter, LinkedIn, SensioLabs Connect, MoiKrug: maxromanovsky Who am I 3
  4. 4. Continuous integration (CI) is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day. http://en.wikipedia.org/wiki/Continuous_integr ation http://www.martinfowler.com/articles/continuo usIntegration.html What is Continuous Integration? 4
  5. 5. • Improve quality of application • Understand current situation on project • Automate your checklists & verifications • Notify different people on different events – Build broken? – Developers – New build deployed to QA? – QA engineers – New build deployed to UAT/Production? – Customers How could it help you? 5
  6. 6. Dependencies • Composer • Platform packages • Phar • Pear • Pecl Build • Phing • phpqatools.org Continuous Integration • Jenkins CI • Additional PHP- related & general plugins • Git Deployment • Package with vendors • Capifony • Doctrine Migrations Our proposal for Symfony-based projects 6
  7. 7. DEPENDENCIES 7
  8. 8. Manages 3rd party dependencies required at runtime* • Bundles • PHP libraries • JS/CSS libraries • *Phpunit for hphpa 8 Composer getcomposer.org
  9. 9. Checklist • stability • lib-*, ext-* • install vs update • require • version masks • Keep composer.phar under VCS • Satis for private packages 9 Composer github.com/composer/satis lib-*, ext-*: goo.gl/Z8dFal
  10. 10. Phar • Only for CLI libraries used for development and build • Composer, php-cs-fixer Pear • Build tools • Phing, phpunit*, phpqatools Pecl • For PHP extensions missing in platform packages Platform packages (apt, yum, ports) • Use it for all purposes if you can (PHP, extensions etc.) Other tools 10
  11. 11. What to update? • Composer dependencies • PHARs • PEARs • PECLs • Platform packages Keep in mind • Update at the beginning of the iteration • Read changelogs • Subscribe to news feeds • Use * as Composer dependency version to receive all updates • Not for Symfony itself  Dev QA Staging • Performance testing • Check logs Production 11 Don’t forget about updates!
  12. 12. BUILD 12
  13. 13. Ant • From Java world • XML declarative config • Can run tasks in parallel • No PHP-specific tasks • Could be extended (Java) • IDE support Phing • Written in PHP • Ant config syntax • All tasks by default are running in the single PHP process • Could be extended (PHP) • IDE support Pake • Written in PHP • Not so popular • PHP-based config • github.com/indeyets/pake/wiki Build tools 13
  14. 14. build-symfony- ci.xml • Default workflow build.xml • Overwrites for current project build.properties.dist • Default configuration build.properties • Project configuration 14 Phing phing.info
  15. 15. Target • A set of tasks • Could depend on other targets • phing <target_name> Task • Action executed as a part of target • Custom tasks can be implemented Types • Reference • FileSet • Property • Other • FileList • Filters Phing Terminology 15
  16. 16. build • for dev environment • outputs to the console build-ci • for CI environment • outputs to XML files package • generates artifacts for deployment • tar.gz archive • SQL schema dump deploy • deploys tar.gz archive to the environment Main Phing Targets 16
  17. 17. Special clean prepare build-tools-self-update Composer- specific composer-validate dependencies-install dependencies-list-updates update –dry-run symfony-security-checker security.sensiolabs.org Phing Targets (1/6) 17
  18. 18. General phplint php -l phploc pdepend charts phpmd + custom config phpcs/phpcs-js + custom config (PSR-1&2, no var_dump(). console.log() etc) phpcpd Phing Targets (2/6) 18
  19. 19. General apidoc PHPDoc projectdoc rst2html phpunit via exec phpcb PHP_CodeBrowser UI php-cs-fixer hphpa HipHop static analysis No target for KPHP yet  Phing Targets (3/6) 19
  20. 20. Symfony- specific copy-configs-ci parameters.yml for CI cache:warmup app/cache is removed during build twig:lint for Bundles and app/Resources assets:install assetic:dump YUICompressor, Google Closure Compressor, lessphp Phing Targets (4/6) 20
  21. 21. Symfony- specific doctrine:database:drop doctrine:database:create doctrine:migrations:migrate doctrine:fixtures:load doctrine:schema:validate doctrine:schema:dump – dump-sql for DDL schema artifact Phing Targets (5/6) 21
  22. 22. Package package-version version.txt package-db-schema schema.sql package-archive Deployment deploy-maintenance- enable/disable deploy-backup-db deploy-clear-doctrine-cache Only if not in APC deploy-cleanup-old-releases deploy-artifact artifact name provided via CLI argument Phing Targets (6/6) 22
  23. 23. CONTINUOUS INTEGRATION 23
  24. 24. CruiseControl • +PHP Under Control TeamCity • Commercial Travis CI • SaaS Scrutinizer CI • SaaS • PHP-oriented Jenkins CI • ex-Hudson CI CI servers 24
  25. 25. • Open-source • Enterprise • Lot of plugins • PHP Template from Sebastian Bergmann • IDE Integration • RESTful API 25 Jenkins CI overview jenkins-ci.org
  26. 26. • No comments git • green & red signs for buildsgreenballs • parameterized builds parameterized- trigger • git tag selector for builds • disable internal tagging! git-parameter • copy artifact from one job to anothercopyartifact • sends emails on successful builds • useful for deployment jobs email-ext 26 Jenkins Plugins (common) goo.gl/IZ2gKE
  27. 27. phing • phpcs • hphpacheckstyle • phpcpddry • php_dependjdepend • phploc using CSV filesplot • phpmdpmd • aggregates info from phpcs, hphpa, phpmd, phpcpdviolations • phpunitxunit • HTML artifactshtmlpublisher 27 Jenkins Plugins (PHP-specific) jenkins-php.org
  28. 28. Master • Bleeding edge • For completed features Feature branch • Merge to master on completion • After code review • Developer executes phing build at least before code review Tag • Stable version from master Bugfix & develop? • Sprint branch • Master branch • Merge feature branches to sprint branch • Jenkins job for sprint branch Git workflow 28
  29. 29. • Builds specific branch (master) • Features are verified after merge build-<branchname> • Parameterized • Produces .tar.gz artifact from specific Git tag build-package-tag Jenkins Jobs for build 29
  30. 30. • Produces .tar.gz artifact from latest revision of branch • Deploys to QA • Not recommended to have more than one such job (issues with migrations) • Workaround: multiple QA or complete DB purge on deploy deploy-qa-<branchname> • Deploys specific package from build-package-tag to specified environment • Ability to enable/disable maintenance mode and error message • Email after deployment deploy-package-tag Jenkins Jobs for deployment 30
  31. 31. Deployed .tar.gz archive • <tag>.<build#> • v0.0.3.26 • <branchname>-<build#>-<revision>-SNAPSHOT • master-12-7c041a90366ce9d958760094c763d809ffbc629e- SNAPSHOT Previous deployed version DB dump before deployment Doctrine Migrations info DB schema Jenkins Jobs for deployment. Artifacts 31
  32. 32. DEPLOYMENT 32
  33. 33. Capistrano • from Ruby world Capifony • Based on Capistrano Shell script • VCS update • Rsync Phar • Don’t use WebPhar • Silex does not use it anymore Zend Server package • No support for vanilla PHP PaaS • AWS Elastic Beanstalk • PagodaBox Deployment tools 33
  34. 34. Based on Capistrano • Supports Capistrano plugins Symfony 1&2 deployment Multistage • qa, staging, uat, production Maintenance mode Clustered environments 34 Capifony capifony.org
  35. 35. Directories • releases/ • current/ (symlink) • shared/ Shared • app/config/parameters.yml • app/cache • app/logs • web/uploads Shared files & dirs are symlink’ed Shared files & dirs are not shared across nodes Directory structure for Capistrano projects 35
  36. 36. `-- /var/www/my-app.com |-- current → /var/www/my-app.com/releases/20100512131539 |-- releases | `-- 20100512131539 | `-- 20100509150741 | `-- 20100509145325 `-- shared |-- web | `-- uploads |-- log `-- config `-- databases.yml Directory structure example 36
  37. 37. All you need is package • capistrano-deploy-strategy-archive Github not available? • Goodbye, vendors! Decreases deployment time • Lot of vendors • Slow internet connection Reproducible • You know what you deploy • You can deploy exactly the same package to multiple environments Deployment via SCM vs package 37
  38. 38. .tar.gz Optimized autoloader Contains only necessary dirs & files assets:install assetic:dump version.txt & db- schema.sql Packaging with Phing 38
  39. 39. DBDeploy Liquibase Doctrine Migrations Doctrine Fixtures Database Migrations 39
  40. 40. User • Triggers job on Jenkins CI • Choose options (env, maintenance mode) Jenkins • Executes single phing target with additional command-line arguments Phing • Executes several Capifony commands according to command-line arguments Capifony • Performs actions on remote machines Capifony & Jenkins CI 40
  41. 41. Enables Maintenance mode • maintenance.html in webroot Backups DB Clears Doctrine cache • Not for APC cache Downloads previous deploy metadata • package version • Doctrine Migrations info Deploys tar.gz archive Capifony & Phing (1/2) 41
  42. 42. Restarts services • php-fpm or Apache • Doctrine APC cache is cleared Symfony-specific tasks • copy app/config/parameters.yml • doctrine:migrations:migrate • cache:warmup Disables Maintenance mode Performs cleanup for old releases Capifony & Phing (2/2) 42
  43. 43. Semi- automatical • from deployment server • usually CI server Enable Maintenance • manually via phing Manually backup for analysis • DB • current version • Doctrine Migrations status • codebase Rollback (1/2) 43
  44. 44. Switch to previous release • Manually via capifony CLI • Recover DB • Rollback Doctrine Migrations • Restore DB dump • Custom Additional tasks • Restore web server config • Restart service • cache:warmup Disable Maintenance • Manually via phing Rollback (2/2) 44
  45. 45. Write rollback guide • Step-by-step • Verify it periodically Enable monitoring on prod • Zabbix • Munin • New Relic • Pinba • Graylog2 Rollback tips 45
  46. 46. SUMMARY 46
  47. 47. • Currently not available online  • Planning to release in 1-2 months on GitHub Where to download? 47
  48. 48. Deployment •Cronjob / CLI script handling •Web server (Apache, nginx) config manipulations •Ext- and lib- dependency verification via Composer What is not implemented yet 48
  49. 49. Questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×