Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Continous Delivering a PHP application

2,318 views

Published on

For the last few months we've been implementing a Continuous Delivery pipeline for the redesign of Time Out. In this talk I will demonstrate a real life example of what our pipeline looks like, the different tools we've used to get it done (phing, github, jenkins, ansible, AWS S3, ...), and peculiarities for PHP and Symfony2 projects. Most importantly, I'll be looking at things we've struggled with along the way and the lessons we've learnt.

Published in: Technology
  • Be the first to comment

Continous Delivering a PHP application

  1. 1. APPLICATIONS CONTINUOUSLY! DELIVERING
  2. 2. Javier López @loalf Senior Platform Engineer @Rightster! formerly Software Architect @TimeOut Certified Symfony Developer Zend Certified PHP Co-organizer @desymfony
  3. 3. DISCLAIMER
  4. 4. AGENDA CONTINUOUS DELIVERY IN A NUTSHELL BUILDING THE PIPELINE BEFORE AND AFTER 1 2 3
  5. 5. CONTINUOUS DELIVERY … … IN A NUTSHELL 1
  6. 6. COMMIT BUILD TEST DEPLOY When and how a change in your code is going to trigger the pipeline
  7. 7. COMMIT BUILD TEST DEPLOY Check the integrity of your code. Produce an artifact.
  8. 8. COMMIT BUILD TEST DEPLOY Check the integrity of your product. ! After this step you should be 100% confident that you could deploy your artifact.
  9. 9. COMMIT BUILD TEST DEPLOY Does what it says on the tin. Triggering this step should be “a click away”. Manual
  10. 10. THE BOOK
  11. 11. THE PIPELINE BUILDING 2
  12. 12. TOOLS! OF THE TRADE
  13. 13. Github plugin https://wiki.jenkins-ci.org/display/JENKINS/GitHub+Plugin Build Pipeline Plugin https://wiki.jenkins-ci.org/display/JENKINS/Build+Pipeline+Plugin
  14. 14. BUILDING! THE PIPELINE
  15. 15. COMMIT BUILD TEST DEPLOY master 6ebb017
  16. 16. COMMIT BUILD TEST DEPLOY master featureA 6ebb017
  17. 17. COMMIT BUILD TEST DEPLOY master featureA 6ebb017 PULL REQUEST
  18. 18. COMMIT BUILD TEST DEPLOY master featureA 6ebb017 9046c48
  19. 19. COMMIT BUILD TEST DEPLOY master featureA 6ebb017 9046c48 BUILD TEST DEPLOY
  20. 20. COMMIT BUILD TEST DEPLOY
  21. 21. 1. Check out master branch COMMIT BUILD TEST DEPLOY
  22. 22. 1. Check out master branch 2. Generate parameters.ini for each environment COMMIT BUILD TEST DEPLOY
  23. 23. 1. Check out master branch 2. Generate parameters.ini for each environment 3. Fetch PHP dependencies COMMIT BUILD TEST DEPLOY
  24. 24. 1. Check out master branch 2. Generate parameters.ini for each environment 3. Fetch PHP dependencies COMMIT BUILD TEST DEPLOY —optimizer-­‐autoload  —prefer-­‐dist
  25. 25. 1. Check out master branch 2. Generate parameters.ini for each environment 3. Fetch PHP dependencies COMMIT BUILD TEST DEPLOY
  26. 26. 1. Check out master branch 2. Generate parameters.ini for each environment 3. Fetch PHP dependencies 4. Run PHPUnit tests COMMIT BUILD TEST DEPLOY
  27. 27. 1. Check out master branch 2. Generate parameters.ini for each environment 3. Fetch PHP dependencies 4. Run PHPUnit tests 5. Fetch JS dependencies COMMIT BUILD TEST DEPLOY
  28. 28. 1. Check out master branch 2. Generate parameters.ini for each environment 3. Fetch PHP dependencies 4. Run PHPUnit tests 5. Fetch JS dependencies 6. Generate assets (JS, CSS) COMMIT BUILD TEST DEPLOY
  29. 29. 1. Check out master branch 2. Generate parameters.ini for each environment 3. Fetch PHP dependencies 4. Run PHPUnit tests 5. Fetch JS dependencies 6. Generate assets (JS, CSS) 7. Generate artifact (zip file) COMMIT BUILD TEST DEPLOY
  30. 30. 1. Check out master branch 2. Generate parameters.ini for each environment 3. Fetch PHP dependencies 4. Run PHPUnit tests 5. Fetch JS dependencies 6. Generate assets (JS, CSS) 7. Generate artifact (zip file) COMMIT BUILD TEST DEPLOY discard unnecessary files
  31. 31. 1. Check out master branch 2. Generate parameters.ini for each environment 3. Fetch PHP dependencies 4. Run PHPUnit tests 5. Fetch JS dependencies 6. Generate assets (JS, CSS) 7. Generate artifact (zip file) COMMIT BUILD TEST DEPLOY
  32. 32. 1. Check out master branch 2. Generate parameters.ini for each environment 3. Fetch PHP dependencies 4. Run PHPUnit tests 5. Fetch JS dependencies 6. Generate assets (JS, CSS) 7. Generate artifact (zip file) 8. Upload artifact to S3 COMMIT BUILD TEST DEPLOY
  33. 33. 1. Check out master branch 2. Generate parameters.ini for each environment 3. Fetch PHP dependencies 4. Run PHPUnit tests 5. Fetch JS dependencies 6. Generate assets (JS, CSS) 7. Generate artifact (zip file) 8. Upload artifact to S3 COMMIT BUILD TEST DEPLOYbuild.xml
  34. 34. <?xml  version="1.0"  encoding="UTF-­‐8"  ?>   <project  name="Time  Out  Miyagi"  description="Time  Out  Website  V4"  default="build"     !    <!-­‐-­‐  Install  PHP  dependencies  (composer)  -­‐-­‐>      <target  name="build:php-­‐dependencies"  depends="build:params">          <echo  msg="Installing  PHP  dependencies  (composer  install)"  />          <exec  command="/opt/composer/composer.phar  install  -­‐-­‐optimize-­‐autoloader  -­‐-­‐prefer-­‐dist"                        logoutput="true"                        checkreturn=“true"          />      </target>   !    <!-­‐-­‐  Test  JS  &  Generate  JS/CSS  assets  (grunt)  -­‐-­‐>      <target  name="build:frontend-­‐dependencies">          <echo  msg="Test  JS  and  Generate  JS/CSS  assets  (grunt)"  />          <exec  command="npm  install"                        logoutput="true"  checkreturn="true"  dir="./web-­‐src"  />          <exec  command="xvfb-­‐run  grunt  -­‐-­‐env=dist"                        logoutput="true"  checkreturn="true"  dir="./web-­‐src"  />          </target>       </project> build.xml
  35. 35. COMMIT BUILD TEST DEPLOY Split into two jobs in Jenkins 1. Test deployment script 2. Run automated tests
  36. 36. COMMIT BUILD TEST DEPLOY DEPLOYMENT SCRIPT
  37. 37. COMMIT BUILD TEST DEPLOY DEPLOYMENT SCRIPT 1. SSH to QA server
  38. 38. COMMIT BUILD TEST DEPLOY DEPLOYMENT SCRIPT 1. SSH to QA server 2. Fetch artifact from S3
  39. 39. COMMIT BUILD TEST DEPLOY DEPLOYMENT SCRIPT 1. SSH to QA server 2. Fetch artifact from S3 3. Unzip artifact
  40. 40. COMMIT BUILD TEST DEPLOY DEPLOYMENT SCRIPT 1. SSH to QA server 2. Fetch artifact from S3 3. Unzip artifact 4. Set right permissions
  41. 41. COMMIT BUILD TEST DEPLOY DEPLOYMENT SCRIPT 1. SSH to QA server 2. Fetch artifact from S3 3. Unzip artifact 4. Set right permissions 5. Update virtual host
  42. 42. COMMIT BUILD TEST DEPLOY DEPLOYMENT SCRIPT 1. SSH to QA server 2. Fetch artifact from S3 3. Unzip artifact 4. Set right permissions 5. Update virtual host 6. Reload PHP-FPM and nginx
  43. 43. COMMIT BUILD TEST DEPLOY DEPLOYMENT SCRIPT 1. SSH to QA server 2. Fetch artifact from S3 3. Unzip artifact 4. Set right permissions 5. Update virtual host 6. Reload PHP-FPM and nginx deploy.yml
  44. 44. -­‐-­‐-­‐   -­‐  hosts:  qa:beta:prod      sudo:  true      vars:          amazon_bucket  :  "releases-­‐miyagi"          base_dir            :  "/var/www/v4.timeout.com"          project_dir      :  "{{  base_dir  }}/{{  git_hash  }}"          web_dir              :  "{{  project_dir  }}/web"      tasks:          -­‐  name:  Fetch  artefact  from  S3              s3:  bucket={{  amazon_bucket  }}  aws_access_key={{  amazon_key  }}   aws_secret_key={{  amazon_secret  }}  object={{  git_hash  }}.gzip  dest="{{  project_dir   }}/{{artefact_zip}}"  mode="get"   !        -­‐  name:  Uncompress  zip  file              command:  chdir="{{  project_dir  }}"  tar  -­‐xf  {{  artefact_zip  }}   !        -­‐  name:  Apply  right  permissions  to  project  root              file:  dest="{{  project_dir  }}"  state=directory  mode=0755  group=nginx   owner=nginx  recurse=true                    -­‐  name:  Restart  PHP-­‐FPM                service:  name=php-­‐fpm  state=reloaded deploy.yml
  45. 45. COMMIT BUILD TEST DEPLOY RUN AUTOMATED TESTS
  46. 46. COMMIT BUILD TEST DEPLOY RUN AUTOMATED TESTS 1. Check out 9046c48
  47. 47. COMMIT BUILD TEST DEPLOY RUN AUTOMATED TESTS 1. Check out 9046c48 2. Fetch PHP dependencies
  48. 48. COMMIT BUILD TEST DEPLOY RUN AUTOMATED TESTS 1. Check out 9046c48 2. Fetch PHP dependencies 3. Run Behat against QA box
  49. 49. COMMIT BUILD TEST DEPLOY Split into two jobs in Jenkins 1. Run deployment script (production) 2. Run automated tests against production
  50. 50. BUILD DEPLOY AUTOMATED! TESTS 6ebb017 6ebb017 QA 6ebb017 QA DEPLOY AUTOMATED! TESTS 6ebb017 PROD 6ebb017 PROD 6ebb017 6ebb017.zip
  51. 51. BUILD DEPLOY AUTOMATED! TESTS 6ebb017 6ebb017 QA 6ebb017 QA DEPLOY AUTOMATED! TESTS 6ebb017 STAGING 6ebb017 STAGING DEPLOY AUTOMATED! TESTS 6ebb017 PROD 6ebb017 PROD 6ebb017 6ebb017.zip SAFETY NET
  52. 52. TIME
  53. 53. BUILD DEPLOY AUTOMATED! TESTS ~3 mins ~30 secs ~5 mins
  54. 54. VISIBILITY EVERYBODY IN THE TEAM SHOULD BE AWARE OF THE STATUS OF THE PIPELINE
  55. 55. BUILD DEPLOY AUTOMATED! TESTS 9046c48 9046c48 QA 9046c48 QA 9046c48 BUILD DEPLOY AUTOMATED! TESTS 6ebb017 6ebb017 QA 6ebb017 QA 6ebb017 BUILD DEPLOY AUTOMATED! TESTS b0b325 b0b325 QA b0b325 QA b0b325 BUILD DEPLOY AUTOMATED! TESTS 99e6d6 99e6d6 QA 99e6d6 QA 99e6d6
  56. 56. http://www.frisnit.com/internet-of-things-ci-status-lamp/
  57. 57. TRACEABILITY EVERYBODY SHOULD KNOW WHAT VERSION IS DEPLOYED IN WHICH ENVIRONMENT
  58. 58. curl  -­‐I  http://www.timeout.com/las-­‐vegas   ! HTTP/1.1  200  OK   Server:  nginx/1.4.7   Vary:  Accept-­‐Encoding   Cache-­‐Control:  no-­‐cache   Content-­‐Type:  text/html;  charset=UTF-­‐8   Date:  Fri,  19  Sep  2014  06:07:29  GMT   Transfer-­‐Encoding:  chunked   Access-­‐Control-­‐Allow-­‐Origin:  http://media.timeout.com   Connection:  Keep-­‐Alive   X-­‐TIMEOUT-­‐V:  d645127afb423e543d90ab5a7b8eae94f248b137   X-­‐Powered-­‐By:  PHP/5.5.14 VERSION NUMBER
  59. 59. https://github.com                /your-­‐company                /your-­‐application                /commits                /your-­‐git-­‐hash
  60. 60. ROLLING! BACK
  61. 61. AUTOMATION
  62. 62. YOU’LL NEED! ALLIES
  63. 63. YOUR ALLIES
  64. 64. DEV TEAM YOUR ALLIES
  65. 65. DEV TEAM QA TEAM YOUR ALLIES
  66. 66. DEV TEAM QA TEAM DEVOPS YOUR ALLIES
  67. 67. DEV TEAM QA TEAM DEVOPS PRODUCT YOUR ALLIES
  68. 68. DEV TEAM QA TEAM DEVOPS PRODUCT THE TOP BRASS YOUR ALLIES
  69. 69. BEFORE AFTER& 3
  70. 70. ONE QA BOX PER FEATURE SAME QA BOX FOR EVERYONE BEFORE AFTER
  71. 71. ~ !WEEKS FROM DEVELOPMENT TO RELEASE ~ !DAYS FROM DEVELOPMENT TO RELEASE BEFORE AFTER
  72. 72. 5 PEOPLE TO RELEASE TO PRODUCTION 1 PERSON TO RELEASE TO PRODUCTION BEFORE AFTER
  73. 73. ~30 MINUTES TO RUN DEPLOYMENT SCRIPT ~30 SECONDS TO RUN DEPLOYMENT SCRIPT BEFORE AFTER
  74. 74. RELEASING WAS AN EVENT RELEASING NOW A NON-EVENT BEFORE AFTER
  75. 75. @loalf

×