Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

SymfonyCon Madrid 2014 - Rock Solid Deployment of Symfony Apps

4,786 views

Published on

Web applications are becoming increasingly more complex, so deployment is not just transferring files with FTP anymore. We will go over the different challenges and how to deploy our PHP applications effectively, safely and consistently with the latest tools and techniques. We will also look at tools that complement deployment with management, configuration and monitoring.

Published in: Technology

SymfonyCon Madrid 2014 - Rock Solid Deployment of Symfony Apps

  1. 1. Photo: Alvaro Videla Rock Solid Deployment of Symfony Applications Pablo Godel @pgodel Photo by @old_sound https://joind.in/12960
  2. 2. Hi, I am Pablo.
  3. 3. Hi, I am Pablo. ! @pgodel
  4. 4. A GUIDING LIGHT FOR STUDENT SUCCESS
  5. 5. A GUIDING LIGHT FOR STUDENT SUCCESS Symfony2 + AngularJS + much more!
  6. 6. A GUIDING LIGHT FOR STUDENT SUCCESS We are Hiring!
  7. 7. Deployment ?
  8. 8. Deployment Software deployment is all of the activities that ! make a software system available for use. http://en.wikipedia.org/wiki/Software_deployment
  9. 9. Deployment A very important part of the application life-cycle
  10. 10. Deployment A very important critical part of the application life-cycle
  11. 11. Deployment It should not be an ! after thought
  12. 12. Deployment It should be predictable
  13. 13. Deployment The more you do it the better it goes
  14. 14. Deployment in the PaaS era
  15. 15. 1. Open an account
  16. 16. 2. Create app
  17. 17. 3. Push code
  18. 18. 4. Voilà!
  19. 19. composer.json ! ! Procfile
  20. 20. $ heroku create!
  21. 21. $ git push heroku master!
  22. 22. $ heroku ps:scale web=5!
  23. 23. That’s it! ;-)
  24. 24. Questions? Slides: http://slideshare.net/pgodel Twitter: @pgodel
  25. 25. But…
  26. 26. Deployment for the rest of us
  27. 27. Deployment Goals
  28. 28. Deployment Goals One-click deploys
  29. 29. Deployment Goals One-click deploys
  30. 30. Deployment Goals Continuous deploys
  31. 31. Deployment Goals Continuous deploys
  32. 32. Deployment Goals Anytime & Anywhere
  33. 33. Deployment Goals Anytime & Anywhere
  34. 34. Anyone Deployment Goals
  35. 35. Anyone Deployment Goals
  36. 36. Reliable Deployment Goals
  37. 37. Reliable Deployment Goals
  38. 38. Rollbacks Deployment Goals
  39. 39. Rollbacks Deployment Goals
  40. 40. No downtime Deployment Goals
  41. 41. No downtime Deployment Goals
  42. 42. Reusable Deployment Goals
  43. 43. Reusable Deployment Goals
  44. 44. Reusable Deployment Goals
  45. 45. Scalable Deployment Goals
  46. 46. Scalable Deployment Goals
  47. 47. Deployment Facts
  48. 48. Deployment Fact #1 Deployment starts with the developer
  49. 49. Deployment Fact #1 Deployment starts with the developer • Development environment must be as close as possible to productions servers!
  50. 50. Deployment Fact #1 Deployment starts with the developer • Development environment must be as close as possible to productions servers! • Setup test/qa/staging servers!
  51. 51. Deployment Fact #1 Deployment starts with the developer • Development environment must be as close as possible to productions servers! • Setup test/qa/staging servers! • Use Vagrant to manage VMs!
  52. 52. Deployment Fact #1 Deployment starts with the developer • Development environment must be as close as possible to productions servers! • Setup test/qa/staging servers! • Use Vagrant to manage VMs! • Use Containers with Docker!
  53. 53. Deployment Fact #1 Deployment starts with the developer • Development environment must be as close as possible to productions servers! • Setup test/qa/staging servers! • Use Vagrant to manage VMs! • Use Containers with Docker! • Automate your development environment setup (Ansible, Homebrew Cask)
  54. 54. Deployment Fact #2 Success depends on server OS setup
  55. 55. Deployment Fact #2 Success depends on server OS setup • Automate installation/configuration with Ansible/ Puppet/Chef !
  56. 56. Deployment Fact #2 Success depends on server OS setup • Automate installation/configuration with Ansible/ Puppet/Chef ! • Create OS packages for 3rd party software
  57. 57. Deployment Fact #2 Success depends on server OS setup • Automate installation/configuration with Ansible/ Puppet/Chef ! • Create OS packages for 3rd party software! • Setup your own package (rpm/deb) repositories
  58. 58. Deployment Fact #3 Monitoring is uptime
  59. 59. Deployment Fact #3 Monitoring is uptime • Use tools to know what is going on with your servers (Ganglia, Cacti, Zabbix, etc.)!
  60. 60. Deployment Fact #3 Monitoring is uptime • Use tools to know what is going on with your servers (Ganglia, Cacti, Zabbix, etc.)!
  61. 61. Deployment Fact #3 Monitoring is uptime • Use tools to know what is going on with your servers (Ganglia, Cacti, Zabbix, etc.)! • Add metrics to your app (Graphite, StatsD, New Relic)!
  62. 62. Deployment Fact #3 Monitoring is uptime • Use tools to know what is going on with your servers (Ganglia, Cacti, Zabbix, etc.)! • Add metrics to your app (Graphite, StatsD, New Relic)! • Use your logs wisely (Logstash, Hecka, Kibana)
  63. 63. Deployment Methodologies
  64. 64. Deployment Methodologies • VIM-style
  65. 65. Deployment Methodologies • VIM-style! • FTP uploads
  66. 66. Deployment Methodologies • VIM-style! • FTP uploads
  67. 67. Deployment Methodologies • VIM-style! • FTP uploads! • Rsync
  68. 68. Deployment Methodologies • VIM-style! • FTP uploads! • Rsync! • Source control (GIT, SVN)
  69. 69. Deployment Methodologies • VIM-style! • FTP uploads! • Rsync! • Source control (GIT, SVN)! • Build tools (Ant, Phing, Jenkins)!
  70. 70. Deployment Methodologies • VIM-style! • FTP uploads! • Rsync! • Source control (GIT, SVN)! • Build tools (Ant, Phing, Jenkins)! • Package based (RPM, DEB, …) !
  71. 71. Deployment Methodologies • VIM-style! • FTP uploads! • Rsync! • Source control (GIT, SVN)! • Build tools (Ant, Phing, Jenkins)! • Package based (RPM, DEB, …) ! • Specialized tools (Capifony, Deployer, …)!
  72. 72. Deployment Methodologies • VIM-style! • FTP uploads! • Rsync! • Source control (GIT, SVN)! • Build tools (Ant, Phing, Jenkins)! • Package based (RPM, DEB, …) ! • Specialized tools (Capifony, Deployer, …)! • IT automation tools (Ansible, Puppet, Chef, …)
  73. 73. Deployment: Steps overview
  74. 74. Deployment: First time • Copy files to server(s)! • Set server-side configurations! • Load DB fixtures! • Process and install assets! • Warm up cache! • Enable site
  75. 75. Deployment: Subsequent times • Copy files to server(s)! • Update server-side configurations! • Update DB (migrations)! • Process and install assets! • Warm up cache! • Enable new version
  76. 76. We can automate!
  77. 77. Deployment Challenges
  78. 78. Deployment Challenges Challenge: Secure
  79. 79. Deployment Challenges Challenge: Secure • use ssh based connections!
  80. 80. Deployment Challenges Challenge: Secure • use ssh based connections! • don’t store passwords on source control
  81. 81. Deployment Challenges Challenge: Secure • use ssh based connections! • don’t store passwords on source control! • use environment variables for sensitive data
  82. 82. Deployment Challenges Challenge: Static assets
  83. 83. Deployment Challenges Challenge: Static assets • Minimize/combine JS and CSS files!
  84. 84. Deployment Challenges Challenge: Static assets • Minimize/combine JS and CSS files! • Enable web server compression!
  85. 85. Deployment Challenges Challenge: Static assets • Minimize/combine JS and CSS files! • Enable web server compression! • Add versioning to static assets links (code.js?v=1)!
  86. 86. Deployment Challenges Challenge: Static assets • Minimize/combine JS and CSS files! • Enable web server compression! • Add versioning to static assets links (code.js?v=1)! • Run utilities locally or in a staging server, create artifact, deploy result!
  87. 87. Deployment Challenges Challenge: Static assets • Minimize/combine JS and CSS files! • Enable web server compression! • Add versioning to static assets links (code.js?v=1)! • Run utilities locally or in a staging server, create artifact, deploy result! • Serve static files from subdomain / CDN
  88. 88. Deployment Challenges Challenge: File permission conflicts
  89. 89. Deployment Challenges Challenge: File permission conflicts • Run Apache/PHP with same user!
  90. 90. Deployment Challenges Challenge: File permission conflicts • Run Apache/PHP with same user! • Use php-fpm instead of mod_php!
  91. 91. Deployment Challenges Challenge: File permission conflicts • Run Apache/PHP with same user! • Use php-fpm instead of mod_php! • Create “deploy” user and add web server to the group
  92. 92. Deployment Challenges Challenge: File permission conflicts • Run Apache/PHP with same user! • Use php-fpm instead of mod_php! • Create “deploy” user and add web server to the group! • Use setfacl to give write access to multiple users!
  93. 93. Deployment Challenges Challenge: File permission conflicts • Run Apache/PHP with same user! • Use php-fpm instead of mod_php! • Create “deploy” user and add web server to the group! • Use setfacl to give write access to multiple users! • Use external services for writing files and create readonly installations
  94. 94. Common Pitfalls
  95. 95. Common Pitfalls • Case sensitive filesystems!
  96. 96. Common Pitfalls • Case sensitive filesystems! ! src/Acme/ClassName ! != ! src/Acme/Classname.php
  97. 97. Common Pitfalls • Case sensitive filesystems! ! src/Acme/ClassName ! != ! src/Acme/Classname.php
  98. 98. Common Pitfalls • Case sensitive filesystems! • Configuration differences!
  99. 99. Common Pitfalls • Case sensitive filesystems! • Configuration differences! • Outdated 3rd party software
  100. 100. Common Pitfalls • Case sensitive filesystems! • Configuration differences! • Outdated 3rd party software! • Github down
  101. 101. Common Pitfalls • Case sensitive filesystems! • Configuration differences! • Outdated 3rd party software! • Github down $ git daemon --base-path=/git/repo/path/ -- export-all! ! $ git clone git://127.0.0.1/repo! http://ozmm.org/posts/when_github_goes_down.html
  102. 102. Common Pitfalls • Case sensitive filesystems! • Configuration differences! • Outdated 3rd party software! • Github down
  103. 103. Deployment Examples
  104. 104. Deployment Examples Simplest continuous deployment ever! hook.php <?php ! ! exec(‘/usr/bin/env -i HOME=/var/www git pull’);! echo “All done!”;
  105. 105. Simplest continuous deployment ever! hook.php Deployment Examples <?php ! ! exec(‘/usr/bin/env -i HOME=/var/www git pull’);! echo “All done!”; GitHub hook screenshot
  106. 106. Deployment Examples Capistrano! ! + ! ! Capifony
  107. 107. Deployment Examples Capistrano / Capifony • Ruby based! • Very extensible! • Large number of extensions! • Simple client side installation
  108. 108. Deployment Examples Capistrano / Capifony • Ruby based! • Very extensible! • Large number of extensions! • Simple client side installation $ gem install capifony!
  109. 109. Deployment Examples Capistrano / Capifony set :application, "myapp" # Application name set :deploy_to, "/var/www/myapp" ! set :user, "deployer" set :use_sudo, false # sudo isn't required ! set :deploy_via, :remote_cache set :repository, "git@github.com:user/repo.git" ! role :web, "server.example.com", “server2.example.com”
  110. 110. Deployment Examples Capistrano / Capifony $ cap deploy:setup
  111. 111. Deployment Examples Capistrano / Capifony ! |-- releases! `-- shared! |-- logs! `-- uploads!
  112. 112. Deployment Examples Capistrano / Capifony $ cap deploy! $ cap deploy:migrations! $ cap deploy:rollback!
  113. 113. Deployment Examples Capistrano / Capifony ! |-- current (symlink to releases/20130112)! |-- releases! | `-- 20130112! `-- shared! |-- logs! `-- uploads!
  114. 114. Deployment Examples Deploying with
  115. 115. Deployment Examples ! ! Define hosts in inventory file [webservers] foo.example.com bar.example.com www[01:50].example.com ! [dbservers] one.example.com two.example.com three.example.com
  116. 116. Deployment Examples ! ! Define tasks / actions in playbook --- - hosts: webservers vars: http_port: 80 max_clients: 200 remote_user: root tasks: - name: ensure apache is at the latest version yum: pkg=httpd state=latest - name: write the apache config file template: src=/srv/httpd.j2 dest=/etc/httpd.conf notify: - restart apache - name: ensure apache is running
  117. 117. Deployment Examples ! ! Reuse tasks with Roles - hosts: webservers roles: - { role: servergrove.symfony2, symfony2_project_root: /var/www/vhosts/ example.com/, symfony2_project_name: demo, symfony2_project_branch: master , symfony2_project_release: 1 }
  118. 118. Deployment Examples ! ! Run process $ ansible-playbook -i inventories/servers playbook.yml -v
  119. 119. Deployment Examples ! ! Run process
  120. 120. Deployment: Other options • Fabric (Python) • Idephix (PHP) • Magellanes (PHP) • Deployer (PHP) • Laravel / envoy (PHP) • Rocketeer (PHP)
  121. 121. Deployment Complement Tools
  122. 122. Deployment Complement Tools Packaging: fpm Build packages for multiple platforms (deb, rpm, etc) with great ease and sanity. $ fpm -s dir -t rpm -n "myapp" -v 1.0 /var/www/myapp ! $ fpm -s dir -t deb -a all -n myapp -v 1.0 /etc/apache2/ conf.d/my.conf /var/www/myapp https://github.com/jordansissel/fpm
  123. 123. Deployment Complement Tools App Metrics: StatsD & Graphite
  124. 124. Deployment Complement Tools App Metrics: liuggio/statsd-php-client $ composer require liuggio/statsd-php-client
  125. 125. Deployment Complement Tools App Metrics: liuggio/statsd-php-client $sender = new SocketSender(/*'localhost', 8126, 'udp'*/); ! $client = new StatsdClient($sender); $factory = new StatsdDataFactory('LiuggioStatsdClientEntityStatsdData'); ! // create the data with the factory $data[] = $factory->timing('usageTime', 100); $data[] = $factory->increment('visitor'); $data[] = $factory->decrement('click'); $data[] = $factory->gauge('gaugor', 333); $data[] = $factory->set('uniques', 765); ! // send the data as array or directly as object $client->send($data);
  126. 126. Deployment Complement Tools App Metrics: liuggio/statsd-php-client $data[] = $factory->timing('usageTime', 100);! ! // send the data as array or directly as object $client->send($data);
  127. 127. Deployment Complement Tools App Metrics: liuggio/statsd-php-client $data[] = $factory->increment('orders');! ! // send the data as array or directly as object $client->send($data);
  128. 128. Deployment Complement Tools App Metrics: liuggio/statsd-php-client $data[] = $factory->gauge('memory', 333);! ! // send the data as array or directly as object $client->send($data);
  129. 129. Deployment Complement Tools App Metrics: StatsD & Graphite App App App StatsD Graphite Grafana
  130. 130. Deployment Complement Tools App Metrics: Grafana
  131. 131. Deployment Complement Tools App Metrics: Grafana
  132. 132. Deployment Complement Tools Logging: Logstash Ship logs from any source, parse them, get the right timestamp, index them, and search them input { file { path => “/var/log/apache2/access_log” } } output { elasticsearch { host => localhost } }
  133. 133. Deployment Complement Tools Logging: Logstash Configure Apache to log json LogFormat "{ "@timestamp": "%{%Y-%m-%dT %H:%M:%S%z}t", "@fields": { "client": "%a", "duration_usec": %D, "status": %s, "request": "%U%q", "method": "%m ", "referrer": "%{Referer}i" } }" logstash_json! ! ! # Write our 'logstash_json' logs to logs/ access_json.log! CustomLog logs/access_json.log logstash_json!
  134. 134. Deployment Complement Tools Logging: Logstash Configure Apache to log json LogFormat "{ "@timestamp": "%{%Y-%m-%dT%H:%M:%S%z}t", "@fields": { "client": "%a", "duration_usec": %D, "status": %s, "request": "%U%q", "method": "%m", "referrer": "% {Referer}i" } }" logstash_json! !! # Write our 'logstash_json' logs to logs/access_json.log! CustomLog logs/access_json.log logstash_json! Result { "@timestamp": "2012-08-22T14:35:19-0700", "client": "127.0.0.1", "duration_usec": 532, "status": 404, "request": "/favicon.ico", "method": "GET", "referrer": "-" }!
  135. 135. Deployment Complement Tools Logging: Kibana Kibana is a user friendly way to view, search and visualize your log data
  136. 136. Deployment Complement Tools Logging: Kibana Kibana is a user friendly way to view, search and visualize your log data
  137. 137. Deployment Complement Tools Logging: Logstash + ElasticSearch + Kibana App / Logstash App / Logstash App / Logstash Redis Logstash filter/processor ElasticSearch Kibana
  138. 138. Deployment Summary
  139. 139. Developer Tests build artifact GitHub Jenkins QA deploy Integration Tests Ansible Production Notifications Deployment Summary
  140. 140. Developer Tests build artifact GitHub Jenkins QA deploy Integration Tests Ansible Production Notifications Deployment Summary
  141. 141. Developer Tests build artifact GitHub Jenkins QA deploy Integration Tests Ansible Production Notifications Deployment Summary
  142. 142. Developer Tests build artifact GitHub Jenkins QA deploy Integration Tests Ansible Production Notifications Deployment Summary
  143. 143. Developer Tests build artifact GitHub Jenkins QA deploy Integration Tests Ansible Production Notifications Deployment Summary
  144. 144. Developer Tests build artifact GitHub Jenkins QA deploy Integration Tests Ansible Production Notifications Deployment Summary
  145. 145. Developer Tests build artifact GitHub Jenkins QA deploy Integration Tests Ansible Production Notifications Deployment Summary
  146. 146. Developer Tests build artifact GitHub Jenkins QA deploy Integration Tests Ansible Production Notifications Deployment Summary
  147. 147. Developer Tests build artifact GitHub Jenkins QA deploy Integration Tests Ansible Production Notifications Deployment Summary
  148. 148. Deployment Summary Developer Tests build artifact GitHub Jenkins QA deploy Integration Tests Ansible Production Notifications
  149. 149. Deployment Summary Developer Tests build artifact GitHub Jenkins QA deploy Integration Tests Ansible Production Notifications
  150. 150. Deployment Future
  151. 151. Deployment Future Containers
  152. 152. Deployment Future Docker Web App Apache / PHP Linux
  153. 153. Deployment Future Docker Host Docker client C VM Web Server C C DB Server App Server
  154. 154. Docker Deployment Future Host Docker client C VM Web Server C C DB Server App Server
  155. 155. Docker Deployment Future Host Docker client C VM Web Server C C DB Server App Server
  156. 156. Docker Deployment Future Host Docker client C VM Web Server C C DB Server App Server Host C VM Web Server C C DB Server App Server
  157. 157. Docker Deployment Future Host Docker client C VM Web Server C C DB Server App Server Host C VM Web Server Host C C DB Server C C App Server C Web Server DB Server App Server Docker Kernel
  158. 158. Final Thoughts
  159. 159. Stop using FTP
  160. 160. Plan early
  161. 161. Practice
  162. 162. Monitor
  163. 163. AUTOMATE!
  164. 164. Now, we did finish! ! Questions?
  165. 165. Feedback Please! https://joind.! in/12960 ! http://slideshare.net/pgodel @pgodel
  166. 166. Feedback Please! https://joind.! in/12960 ! Thank you! http://slideshare.net/pgodel @pgodel

×