Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tame Your Build And Deployment Process With Hudson, PHPUnit, and SSH

Wil Moore III's presentation at the January 20, 2011 Front Range PHP Group.

  • Login to see the comments

Tame Your Build And Deployment Process With Hudson, PHPUnit, and SSH

  2. 2. <ul><ul><li>Senior Developer, Technical Lead @ Navigant Healthcare </li></ul></ul><ul><ul><li>Zend Certified PHP 5.3 Engineer </li></ul></ul><ul><ul><li>Zend Framework Contributor </li></ul></ul><ul><ul><li>Twitter: @wilmoore </li></ul></ul>About Me
  3. 3. <ul><ul><li>Your take-away should be that you understand the steps involved in getting basic continuous integration going. </li></ul></ul><ul><ul><li>You understand that there is no single recipe for success. Each environment is different and Hudson is a single tool. </li></ul></ul><ul><ul><li>You should understand when to use Hudson and when not to use Hudson. </li></ul></ul><ul><ul><li>You should understand that continuous integration isn’t hard. </li></ul></ul>Goal of this talk
  4. 4. <ul><ul><li>What is it and what are the benefits? </li></ul></ul><ul><ul><li>Automates your semi-automated processes and provides instant feedback about these processes. </li></ul></ul><ul><ul><li>Keeps the team honest as it relates to unit and integration tests. </li></ul></ul><ul><ul><li>Yet another method of documenting your process. </li></ul></ul><ul><ul><li>FYI…the name continuous integration sort of sucks. </li></ul></ul>About Continuous Integration
  5. 5. <ul><ul><li>There is nothing magical about continuous integration. </li></ul></ul><ul><ul><li>It won’t write unit tests for you, and there is no single recipe for setting it all up. </li></ul></ul><ul><ul><li>You could automate your infrastructure without a continuous integration server, but you’d be re-inventing the wheel…and you’d still need to understand your process. </li></ul></ul>You need to understand your process
  6. 6. <ul><ul><li>It’s free </li></ul></ul><ul><ul><li>It has a simple and clean interface </li></ul></ul><ul><ul><li>You can also run it from the command-line </li></ul></ul><ul><ul><li>It’s easy to install (especially with Ubuntu) </li></ul></ul>Why Hudson?
  7. 7. So, how do I get started <ul><ul><li>Update Ubuntu > sudo apt-get update && sudo apt-get dist-upgrade –f > sudo apt-get autoremove ; sudo apt-get autoclean </li></ul></ul><ul><ul><li>Install Vim (or your favorite text editor) and SSH > sudo apt-get install vim ssh –y > sudo update-alternatives --config editor </li></ul></ul><ul><ul><li>SSH Security (sudo vim /etc/ssh/sshd_config) PermitRootLogin no PubkeyAuthentication yes PermitEmptyPasswords no PasswordAuthentication no (be careful) > sudo service ssh restart </li></ul></ul>
  8. 8. Install Java & Hudson <ul><ul><li>Java (JDK) Installation > sudo sh -c 'echo &quot;deb lucid partner&quot; >> /etc/apt/sources.list.d/ubuntu-partner.list' > sudo apt-get update > sudo apt-get install sun-java6-jdk -y </li></ul></ul><ul><ul><li>Install Hudson > wget -q -O - | sudo apt-key add - > sudo sh -c 'echo &quot;deb binary/&quot; >> /etc/apt/sources.list.d/hudson.list’ > sudo apt-get update > sudo apt-get install hudson –y Hudson will be available @ http://localhost:8080 </li></ul></ul>
  9. 9. <ul><ul><li>Required plug-ins for this talk </li></ul></ul><ul><ul><ul><li>Git (yes, that’s all) </li></ul></ul></ul><ul><ul><li>Additional plug-ins to explore </li></ul></ul><ul><ul><ul><li>xUnit </li></ul></ul></ul><ul><ul><ul><li>Phing </li></ul></ul></ul><ul><ul><ul><li>SSH </li></ul></ul></ul><ul><ul><ul><li>Github </li></ul></ul></ul><ul><ul><ul><li>Green Balls </li></ul></ul></ul><ul><ul><ul><li>Mercurial </li></ul></ul></ul><ul><ul><ul><li>Subversion </li></ul></ul></ul>Configure Hudson
  10. 10. How to Setup a Hudson project <ul><ul><li>New Job (name = Zend Framework Contributions) </li></ul></ul><ul><ul><li>Build a free-style software project </li></ul></ul><ul><ul><li>Source Code Management = “Subversion” </li></ul></ul><ul><ul><li>Repository URL = </li></ul></ul><ul><ul><li>Choose a Build Trigger </li></ul></ul><ul><ul><li>Add a build step </li></ul></ul><ul><ul><ul><li>Execute Shell </li></ul></ul></ul><ul><ul><ul><li>cd tests && phpunit Zend/Rest/RouteTest.php </li></ul></ul></ul><ul><ul><li>Save, then “Build Now” </li></ul></ul><ul><ul><li>Click the progress bar to watch the build logs real-time </li></ul></ul><ul><ul><li>Note: first time pulling this project down could take a while… </li></ul></ul>
  11. 11. <ul><ul><li>It is inspired by Java’s JUnit framework </li></ul></ul><ul><ul><li>Instead of writing code first, then using echo, print_r, var_export, you write tests first, then write code that will pass the tests. </li></ul></ul><ul><ul><li>@dataProvider allows you to pass arbitrary amounts of test data to a single test so you can see multiple permutations of a single test. </li></ul></ul><ul><ul><li>@group allows you to run specific tests or suites as a group. For instance, you can tag a test (0r group of tests) as corresponding to a bug and run only those tests. </li></ul></ul>About PHPUnit
  12. 12. Successful Build
  13. 13. What else should I run besides unit tests? <ul><ul><li>Run database schema migrations (e.g. Doctrine/Liquibase) </li></ul></ul><ul><ul><li>Run your code generation scripts (dynamic proxies, etc.) </li></ul></ul><ul><ul><li>Publish code coverage stats </li></ul></ul><ul><ul><li>Generate and Publish documentation (PHPDoc, Sphinx) </li></ul></ul><ul><ul><li>Run a database backup </li></ul></ul><ul><ul><li>Restart apache </li></ul></ul><ul><ul><li>Switch release directories (symlinks) </li></ul></ul><ul><ul><li>Pre-warm caches (memcached, apc, files, no-sql databases) </li></ul></ul><ul><ul><li>Update Solr/Lucene indexes </li></ul></ul><ul><ul><li>Start an Amazon EC2 server instance </li></ul></ul>
  14. 14. <ul><ul><li>Local VM, DEV, QA, UAT, STAGE, PROD </li></ul></ul><ul><ul><li>Each should be it’s own hudson job </li></ul></ul><ul><ul><li>You can create a “template” job in hudson </li></ul></ul><ul><ul><li>Build DEV automatically upon your source repository changes </li></ul></ul><ul><ul><li>Build QA on a schedule (nightly, etc.), UAT pushes can be run on-demand, and STAGE/PROD are on-demand just before a new release. </li></ul></ul>What about other environments
  15. 15. Let’s write some code <ul><ul><li>cd ~ </li></ul></ul><ul><ul><li>rm -rf ~/Projects/Personal/ </li></ul></ul><ul><ul><li>mkdir -p ~/Projects/Personal/ </li></ul></ul><ul><ul><li>cd !$ </li></ul></ul><ul><ul><li>git init </li></ul></ul><ul><ul><li>mkdir -p src/lib src/put/lib src/www src/app/controllers src/app/models ops/etc/apache2/sites-available </li></ul></ul><ul><ul><li>vim src/www/index.php </li></ul></ul><ul><ul><li>vim ops/etc/apache2/sites-available/ </li></ul></ul><ul><ul><li>git add . </li></ul></ul><ul><ul><li>git commit -a -m “Initial Commit” </li></ul></ul>
  16. 16. <ul><ul><li>Create an SSH user that will own the web files </li></ul></ul><ul><ul><li>Give SSH user permission to restart apache </li></ul></ul><ul><ul><li>Prepare the application releases directory </li></ul></ul><ul><ul><li>Configure Hudson to pull from Git repo and transfer via rsync </li></ul></ul><ul><ul><li>Configure Hudson Build Steps </li></ul></ul>Can we push yet? Why not?
  17. 17. <ul><ul><li>It’s like telnet, but communication is encrypted as opposed to plain text. </li></ul></ul><ul><ul><li>You can authenticate with a password or with public/private key pairs. </li></ul></ul><ul><ul><li>You keep the private key tucked secretly away, and you give the public key away to servers you want to authenticate with. For instance, Github has your public key if you want to push to their service. </li></ul></ul><ul><ul><li>You can login to a remote server’s shell interactively or you can pass commands to that shell non-interactively. </li></ul></ul>The least you need to know about SSH
  18. 18. Create an SSH User on target server <ul><ul><li>NOTE: SSH into target server to run the following commands: </li></ul></ul><ul><ul><li>sudo useradd api --create-home --skel /etc/skel --shell /bin/bash </li></ul></ul><ul><ul><li>sudo addgroup deploy </li></ul></ul><ul><ul><li>sudo usermod -a --groups deploy api </li></ul></ul><ul><ul><li>ssh-keygen -b 2048 -t rsa -f ~/.ssh/api.pem NOTE: press enter a few times until done – do not enter a passphrase </li></ul></ul><ul><ul><li>cd ~/.ssh </li></ul></ul><ul><ul><li>mv </li></ul></ul><ul><ul><li>sudo mkdir /var/lib/hudson/.ssh </li></ul></ul><ul><ul><li>sudo mv api.pem /var/lib/hudson/.ssh </li></ul></ul><ul><ul><li>sudo chown -R hudson:nogroup /var/lib/hudson/.ssh </li></ul></ul><ul><ul><li>sudo chmod 700 /var/lib/hudson/.ssh </li></ul></ul><ul><ul><li>sudo mkdir ~api/.ssh </li></ul></ul><ul><ul><li>sudo mv ~api/.ssh/authorized_keys </li></ul></ul><ul><ul><li>sudo chown -R api:api ~api/.ssh </li></ul></ul><ul><ul><li>sudo chmod 700 ~api/.ssh </li></ul></ul><ul><ul><li>sudo chmod 600 ~api/.ssh/authorized_keys </li></ul></ul>
  19. 19. <ul><ul><li>About the hudson file system structure </li></ul></ul><ul><ul><li>Hudson is an actual OS user with a shell and a home directory: /var/lib/hudson </li></ul></ul><ul><ul><li>/var/lib/hudson/jobs/{{job-name}}/workspace This is where your files go after a source check-out </li></ul></ul><ul><ul><li>The workspace ($WORKSPACE) directory is where you run unit tests before pushing via SSH. </li></ul></ul>Where are my files going?
  20. 20. Grant SSH user specific permissions <ul><ul><li>Allow the “deploy” group to write and own virtual host configuration files </li></ul></ul><ul><ul><li>sudo chgrp deploy /etc/apache2/sites-available /etc/apache2/sites-enabled </li></ul></ul><ul><ul><li>sudo chmod g+w /etc/apache2/sites-available /etc/apache2/sites-enabled </li></ul></ul><ul><ul><li>sudo chmod g+s /etc/apache2/sites-available /etc/apache2/sites-enabled </li></ul></ul><ul><ul><li>EDIT THE &quot;/etc/sudoers&quot; file </li></ul></ul><ul><ul><li>sudo visudo NOTE: Ubuntu may default to nano as editor; however, you can change it to vim or something else with the following command: sudo update-alternatives --config editor </li></ul></ul><ul><ul><li>ADD THE FOLLOWING LINE (under the &quot;User privilege specification&quot; section) </li></ul></ul><ul><ul><li>api ALL=(root) NOPASSWD: /usr/sbin/service apache2 * </li></ul></ul>
  21. 21. Create the releases directory <ul><ul><li>NOTE: Login as “hudson” user, then SSH to “api” user. </li></ul></ul><ul><ul><li>sudo su – hudson </li></ul></ul><ul><ul><li>vim ~/.ssh/config Host User api IdentityFile /var/lib/hudson/.ssh/api.pem Port 22 Protocol 2 </li></ul></ul><ul><ul><li>ssh -i ~/.ssh/api.pem api@ </li></ul></ul><ul><ul><li>mkdir -p ~/apps/ ~/apps/ NOTE: RSA key fingerprint must be accepted once manually in order to automate authentication. </li></ul></ul>
  22. 22. Configure Hudson for Git & Rsync <ul><ul><li>In Shell </li></ul></ul><ul><ul><li>git config --global &quot;Hudson” </li></ul></ul><ul><ul><li>git config --global &quot;; </li></ul></ul><ul><ul><li>Hudson Job Configuration </li></ul></ul><ul><ul><li>“” >> “Configure” >> “Source Code Management” >> “Git” </li></ul></ul><ul><ul><ul><li>URL of Repository: /media/psf/ NOTE: I cheated by linking a folder to my VM </li></ul></ul></ul><ul><ul><li>“ Build” >> “Add Build Step” >> “Execute Shell” >> “Command” </li></ul></ul><ul><ul><li>rsync -avzH -e ssh --progress $WORKSPACE/ api@$BUILD_ID </li></ul></ul>
  23. 23. Configure Build <ul><ul><li>Hudson Job Configuration (cont.) </li></ul></ul><ul><ul><li>“ Build” >> “Add Build Step” >> “Execute Shell” >> “Command” </li></ul></ul><ul><ul><li>ssh api@ <<ssh-session </li></ul></ul><ul><ul><li>########## BEGIN: ssh commands ########## </li></ul></ul><ul><ul><li>cp ~/apps/${BUILD_ID}/ops/etc/apache2/sites-available/ /etc/apache2/sites-available/ </li></ul></ul><ul><ul><li>ln -nfs /etc/apache2/sites-available/ /etc/apache2/sites-enabled/ </li></ul></ul><ul><ul><li>ln -nfs ~/apps/${BUILD_ID} ~/apps/ </li></ul></ul><ul><ul><li>sudo /usr/sbin/service apache2 restart </li></ul></ul><ul><ul><li>########## END: ssh commands ########## </li></ul></ul><ul><ul><li>ssh-session </li></ul></ul>Now, run build, then browse to
  24. 24. <ul><ul><li>A Restful web service API maps HTTP to code </li></ul></ul><ul><ul><li>Responds to common HTTP verbs/methods (GET, POST, PUT, DELETE) </li></ul></ul><ul><ul><li>Exposes a restful (not RPC, not Query) API </li></ul></ul><ul><ul><ul><ul><li>Restful: /users/1 </li></ul></ul></ul></ul><ul><ul><ul><ul><li>RPC: /users/show/1 </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Query: /users/show?userId=1 </li></ul></ul></ul></ul><ul><ul><li>Responds with a JSON content type (application/json) </li></ul></ul>Let’s build something more interesting to deploy
  25. 25. <ul><ul><li>What is a Router class? </li></ul></ul><ul><ul><li>The “Router” class will allow us to match information of $_SERVER[‘REQUEST_URI’] to a controller class. </li></ul></ul><ul><ul><li>The “Router” class will allow us to match information of $_SERVER[‘REQUEST_URI’] to an anonymous function. </li></ul></ul><ul><ul><li>The “Router” class will allow us to match information of $_SERVER[‘REQUEST_URI’] to a global function. </li></ul></ul>Building the routing class
  26. 26. <ul><ul><li>A Restful web service API </li></ul></ul><ul><ul><li>Support output buffering </li></ul></ul><ul><ul><li>Error & Exception Handling </li></ul></ul><ul><ul><li>Route optional segments </li></ul></ul><ul><ul><li>Parameter conditions </li></ul></ul><ul><ul><li>Content-type negotiation </li></ul></ul><ul><ul><li>Correctly respond to HEAD requests </li></ul></ul>Possible Future Enhancements
  27. 27. <ul><ul><li>The Files </li></ul></ul><ul><ul><li>autoload.php.dist </li></ul></ul><ul><ul><li>bootstrap.php </li></ul></ul><ul><ul><li>phpunit.xml.dist </li></ul></ul><ul><ul><li>src/put/lib/RouterTest.php </li></ul></ul><ul><ul><li>src/lib/Router.php </li></ul></ul><ul><ul><li>src/www/index.php </li></ul></ul>Let’s code it together then deploy it
  28. 28. <ul><ul><li>Bamboo </li></ul></ul><ul><ul><li>Team City </li></ul></ul><ul><ul><li>CruiseControl (PHPUnderControl) </li></ul></ul><ul><ul><li>Team Foundation Server </li></ul></ul>Hudson Alternatives
  29. 29. Books <ul><li>Continuous Integration: Improving Software Quality and Reducing Risk Paul M. Duvall, Steve Matyas, Andrew Glover </li></ul><ul><li>Refactoring: improving the design of existing code Martin Fowler </li></ul><ul><li>Refactoring Databases : Evolutionary Database Design Scott W. Ambler, Pramodkumar J. Sadalage </li></ul><ul><li>Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation Jez Humble, David Farley </li></ul><ul><li>The Pragmatic Programmer: From Journeyman to Master Andrew Hunt, David Thomas </li></ul><ul><li>Agile Testing: A Practical Guide for Testers and Agile Teams Lisa Crispin, Janet Gregory </li></ul>
  30. 30. Reference <ul><li>CI Feature Matrix </li></ul><ul><li>Free Hosted Continuous Integration Environment </li></ul><ul><li>Why are you still not using Hudson? </li></ul><ul><li>Continuous Integration (Martin Fowler) </li></ul>