Rock Solid Deployment of Web Applications


Published on

Presentation given at CodeMiami 2013

Rock Solid Deployment of Web Applications

  1. 1. Rock Solid Deploymentof Web Applications Pablo Godel @pgodel - Jan 12th 2013 - Miami, FL
  2. 2. Who Am I?⁃ Born in Argentina, living in the US since 1999⁃ PHP & Symfony developer⁃ Founder of the original PHP mailing list in spanish⁃ Co-founder of ServerGrove
  3. 3. ServerGrove!⁃ Founded ServerGrove Networks in 2005⁃ Provider of web hosting specialized in PHP, Symfony, ZendFramework, MongoDB and others⁃ Servers in USA (right here in Miami) and Europe!
  4. 4. Community is our teacher⁃ Very active open source supporter through code contributions and usergroups/conference sponsoring
  5. 5. Deployment ?
  6. 6. DeploymentSoftware deployment is all of the activities that makea software system available for use.
  7. 7. DeploymentA very important part ofthe application life-cycle
  8. 8. DeploymentA very important critical partof the application life-cycle
  9. 9. DeploymentIt should not be an after thought
  10. 10. DeploymentIt should be predictable
  11. 11. DeploymentThe more you do it the better it goes
  12. 12. Deployment: Goals
  13. 13. Deployment: GoalsOne-click deploys
  14. 14. Deployment: GoalsContinuous deploys
  15. 15. Web AppsDeployment
  16. 16. Web AppsDeployment
  17. 17. Web AppsDeployment
  18. 18. Deployment: Goals Anytime & Anywhere
  19. 19. Deployment: GoalsAnyone
  20. 20. Deployment: Goals Reliable
  21. 21. Deployment: GoalsRollbacks
  22. 22. Deployment: Goals No downtime
  23. 23. Deployment: GoalsReusable
  24. 24. Deployment: Goals Scalable
  25. 25. Deployment: Goals• One-click / continuous deploys• Anytime & Anywhere• Anyone• No downtime• Predictable & Reliable• Rollbacks• Reusable• Scalable
  26. 26. Deployment Facts
  27. 27. Deployment: Fact #1Deployment starts with the developer• Setup development environment to be as close as possible to productions servers• Setup test/qa/staging servers• Use Vagrant to manage VMs• Use Puppet/Chef to manage OS packages/ configuration
  28. 28. Deployment: Fact #2Success linked to server OS setup• Use Puppet/Chef to manage OS packages/ configuration• Create OS packages for 3rd party software• Setup your own package repositories
  29. 29. Deployment: Fact #3Monitoring is uptime• Use monitoring tools to know what is going on with your servers (Ganglia, Cacti, Zabbix, etc.)• Add monitoring and metrics to your app (Graphite, StatsD, New Relic)• Use your logs wisely (Graylog, Logstash, Kibana)
  30. 30. Deployment Methodologies
  31. 31. Deployment Methodologies • VIM-style • FTP uploads • rsync • source control (svn, git) • Build tools (ant, phing) • Specialized tools (capistrano, fabric, etc) • Package based (rpm, deb, etc)
  32. 32. Web Apps Deployment: Steps overview
  33. 33. Web Apps Deployment: First time• Copy files to server(s)• Set server-side configurations• Load DB fixtures• Process and install assets• Warm up cache• “Enable” site
  34. 34. Web Apps Deployment: Subsequent times• Copy files to server(s)• Apply DB updates (migrations)• Process and install assets• Warm up cache• “Enable” site
  35. 35. Deployment: Challenges
  36. 36. Deployment: ChallengesChallenge:Fast & reliable copy of filesSolutions:• rsync• git pull• setup git repo on local network to save bandwidth and avoid issues if git server is down (i.e. github)
  37. 37. Deployment: ChallengesChallenge:ScalableSolutions:• use a tool that allows to go from 1 to n servers easily (i.e. capistrano)• pssh allows to send commands to n servers in parallel• package your app in OS packages like .rpm/.deb to easily install across n servers
  38. 38. Deployment: ChallengesChallenge:RollbacksSolutions:• test!• tag releases• dedicated branches (master for production)• deploy each release in its own directory
  39. 39. Deployment: ChallengesChallenge:SecureSolutions:• use ssh based connections• don’t store passwords on source control• store sensitive strings (passwords) in server environment variables
  40. 40. Deployment: ChallengesChallenge:DB migrationsSolutions:• Doctrine Migrations• Consider document oriented DBs like MongoDB“The best migrations are the ones you don’t have to do”
  41. 41. Deployment: ChallengesChallenge:Static assetsSolutions:• YUICompress shrinks JS and CSS file sizes• Enable web server compression• Add versioning to static assets links (code.js?v=1)• Assetic combines multiple files into one• Run utilities locally or in a staging server, deploy result
  42. 42. Deployment: ChallengesChallenge:CachingSolutions:• Update one server while others handle load• Group servers and update group at a time• execute commands on “finalize” to clear up APC cache
  43. 43. Deployment: ChallengesChallenge:File permission conflictsSolutions:• Run Apache/PHP with same user• Use php-fpm instead of mod_php• Create “deploy” user and add web server to the group• Use setfacl to give write access to multiple users
  44. 44. Web Apps Deployment:Other common pitfalls
  45. 45. Web Apps Deployment: Other common pitfalls• Case sensitive filesystems• Configuration differences• Outdated 3rd party software• Github down$ git daemon --base-path=/git/repo/path/ --export-all$ git clone git://
  46. 46. Web Apps Deployment: Examples
  47. 47. Web Apps Deployment: Examples Simplest continuous deployment ever!hook.php <?php exec(‘/usr/bin/env -i HOME=/var/www git pull’); echo “All done!”; screenshot
  48. 48. Web Apps Deployment: Examples Capistrano• Ruby based• Very extensible• Large number of extensions• Simple client side installation $ gem install capistrano
  49. 49. Web Apps Deployment: Examples Capistranoset :application, "myapp" # Application nameset :deploy_to, "/var/www/myapp"set :user, "deployer"set :use_sudo, false # sudo isnt requiredset :deploy_via, :remote_cacheset :repository, ""role :web, "", “”
  50. 50. Web Apps Deployment: Examples Capistrano $ cap deploy:setup
  51. 51. Web Apps Deployment: Examples Capistrano |-- releases `-- shared |-- logs `-- uploads
  52. 52. Web Apps Deployment: Examples Capistrano $ cap deploy $ cap deploy:migrations $ cap deploy:rollback
  53. 53. Web Apps Deployment: Examples Capistrano |-- current (symlink to releases/20130112) |-- releases | `-- 20130112 `-- shared |-- logs `-- uploads
  54. 54. Web Apps Deployment: Other options• Fabric• WePloy• Phing• Magallanes• Jenkins
  55. 55. Web Apps Deployment: Tools
  56. 56. Web Apps Deployment: Tools App Metrics: StatsD & Graphite
  57. 57. Web Apps Deployment: Tools Logging: LogstashShip logs from any source, parse them, get the righttimestamp, index them, and search them
  58. 58. Web Apps Deployment: Tools Logging: LogstashConfigure Apache to log jsonLogFormat "{ "@timestamp": "%{%Y-%m-%dT%H:%M:%S%z}t", "@fields": { "client": "%a","duration_usec": %D, "status": %s, "request": "%U%q", "method": "%m", "referrer": "%{Referer}i" } }" logstash_json# Write our logstash_json logs to logs/access_json.logCustomLog logs/access_json.log logstash_jsonResult{ "@timestamp": "2012-08-22T14:35:19-0700", "client": "","duration_usec": 532, "status": 404, "request": "/favicon.ico","method": "GET", "referrer": "-" }
  59. 59. Web Apps Deployment: Tools Logging: Graylog
  60. 60. Web Apps Deployment: Tools Logging: KibanaKibana is a user friendly way to view, search and visualizeyour log data
  61. 61. Web Apps Deployment: Tools Packaging: fpmBuild packages for multiple platforms (deb, rpm, etc) with great ease and sanity. fpm -s dir -t rpm -n "myapp" -v 1.0 /var/www/myapp fpm -s dir -t deb -a all -n myapp -v 1.0 /etc/apache2/ conf.d/my.conf /var/www/myapp
  62. 62. Web Apps Deployment: Summary
  63. 63. •Stop using FTP
  64. 64. •Stop using FTP•Plan early
  65. 65. •Stop using FTP•Plan early•Practice
  66. 66. •Stop using FTP•Plan early•Practice•Monitor
  67. 67. •Stop using FTP•Plan early•Practice•Monitor•AUTOMATE!
  68. 68. QUESTIONS?Slides: Twitter: @pgodel E-mail:
  69. 69. Thank you!Slides: Twitter: @pgodel E-mail: