Tame Your Build And Deployment Process With Hudson, PHPUnit, and SSH

"THE BASIC RULE OF THUMB IS THAT YOU SHOULD BE ABLE TO WALK UP TO THE PROJECT WITH A VIRGIN MACHINE, DO A CHECKOUT, AND BE ABLE TO FULLY BUILD THE SYSTEM.” TAME YOUR BUILD AND DEPLOY PROCESS A LOOK AT HUDSON, PHPUNIT, AND SSH WIL MOORE III @WILMOORE HTTP://GITHUB.COM/WILMOORE

Senior Developer, Technical Lead @ Navigant Healthcare http://navigantconsulting.com/industries/healthcare/
Zend Certified PHP 5.3 Engineer
Zend Framework Contributor
Twitter: @wilmoore
About Me

Your take-away should be that you understand the steps involved in getting basic continuous integration going.
You understand that there is no single recipe for success. Each environment is different and Hudson is a single tool.
You should understand when to use Hudson and when not to use Hudson.
You should understand that continuous integration isn’t hard.
Goal of this talk

What is it and what are the benefits?
Automates your semi-automated processes and provides instant feedback about these processes.
Keeps the team honest as it relates to unit and integration tests.
Yet another method of documenting your process.
FYI…the name continuous integration sort of sucks.
About Continuous Integration

There is nothing magical about continuous integration.
It won’t write unit tests for you, and there is no single recipe for setting it all up.
You could automate your infrastructure without a continuous integration server, but you’d be re-inventing the wheel…and you’d still need to understand your process.
You need to understand your process

It’s free
It has a simple and clean interface
You can also run it from the command-line
It’s easy to install (especially with Ubuntu)
Why Hudson?

So, how do I get started
Update Ubuntu > sudo apt-get update && sudo apt-get dist-upgrade –f > sudo apt-get autoremove ; sudo apt-get autoclean
Install Vim (or your favorite text editor) and SSH > sudo apt-get install vim ssh –y > sudo update-alternatives --config editor
SSH Security (sudo vim /etc/ssh/sshd_config) PermitRootLogin no PubkeyAuthentication yes PermitEmptyPasswords no PasswordAuthentication no (be careful) > sudo service ssh restart

Install Java & Hudson
Java (JDK) Installation > sudo sh -c 'echo "deb http://archive.canonical.com/ lucid partner" >> /etc/apt/sources.list.d/ubuntu-partner.list' > sudo apt-get update > sudo apt-get install sun-java6-jdk -y
Install Hudson > wget -q -O - http://pkg.hudson-labs.org/debian/hudson-labs.org.key | sudo apt-key add - > sudo sh -c 'echo "deb http://pkg.hudson-labs.org/debian binary/" >> /etc/apt/sources.list.d/hudson.list’ > sudo apt-get update > sudo apt-get install hudson –y Hudson will be available @ http://localhost:8080

Required plug-ins for this talk
Git (yes, that’s all)
Additional plug-ins to explore
xUnit
Phing
SSH
Github
Green Balls
Mercurial
Subversion
Configure Hudson

How to Setup a Hudson project
New Job (name = Zend Framework Contributions)
Build a free-style software project
Source Code Management = “Subversion”
Repository URL = http://framework.zend.com/svn/framework/standard/trunk
Choose a Build Trigger
Add a build step
Execute Shell
cd tests && phpunit Zend/Rest/RouteTest.php
Save, then “Build Now”
Click the progress bar to watch the build logs real-time
Note: first time pulling this project down could take a while…

It is inspired by Java’s JUnit framework
Instead of writing code first, then using echo, print_r, var_export, you write tests first, then write code that will pass the tests.
@dataProvider allows you to pass arbitrary amounts of test data to a single test so you can see multiple permutations of a single test.
@group allows you to run specific tests or suites as a group. For instance, you can tag a test (0r group of tests) as corresponding to a bug and run only those tests.
About PHPUnit

Successful Build

What else should I run besides unit tests?
Run database schema migrations (e.g. Doctrine/Liquibase)
Run your code generation scripts (dynamic proxies, etc.)
Publish code coverage stats
Generate and Publish documentation (PHPDoc, Sphinx)
Run a database backup
Restart apache
Switch release directories (symlinks)
Pre-warm caches (memcached, apc, files, no-sql databases)
Update Solr/Lucene indexes
Start an Amazon EC2 server instance

Local VM, DEV, QA, UAT, STAGE, PROD
Each should be it’s own hudson job
You can create a “template” job in hudson
Build DEV automatically upon your source repository changes
Build QA on a schedule (nightly, etc.), UAT pushes can be run on-demand, and STAGE/PROD are on-demand just before a new release.
What about other environments

Let’s write some code
cd ~
rm -rf ~/Projects/Personal/example.com.api
mkdir -p ~/Projects/Personal/example.com.api
cd !$
git init
mkdir -p src/lib src/put/lib src/www src/app/controllers src/app/models ops/etc/apache2/sites-available
vim src/www/index.php
vim ops/etc/apache2/sites-available/example.com.api.vhost
git add .
git commit -a -m “Initial Commit”

Create an SSH user that will own the web files
Give SSH user permission to restart apache
Prepare the application releases directory
Configure Hudson to pull from Git repo and transfer via rsync
Configure Hudson Build Steps
Can we push yet? Why not?

It’s like telnet, but communication is encrypted as opposed to plain text.
You can authenticate with a password or with public/private key pairs.
You keep the private key tucked secretly away, and you give the public key away to servers you want to authenticate with. For instance, Github has your public key if you want to push to their service.
You can login to a remote server’s shell interactively or you can pass commands to that shell non-interactively.
The least you need to know about SSH

Create an SSH User on target server
NOTE: SSH into target server to run the following commands:
sudo useradd api --create-home --skel /etc/skel --shell /bin/bash
sudo addgroup deploy
sudo usermod -a --groups deploy api
ssh-keygen -b 2048 -t rsa -f ~/.ssh/api.pem NOTE: press enter a few times until done – do not enter a passphrase
cd ~/.ssh
mv api.pem.pub api.pub
sudo mkdir /var/lib/hudson/.ssh
sudo mv api.pem /var/lib/hudson/.ssh
sudo chown -R hudson:nogroup /var/lib/hudson/.ssh
sudo chmod 700 /var/lib/hudson/.ssh
sudo mkdir ~api/.ssh
sudo mv api.pub ~api/.ssh/authorized_keys
sudo chown -R api:api ~api/.ssh
sudo chmod 700 ~api/.ssh
sudo chmod 600 ~api/.ssh/authorized_keys

About the hudson file system structure
Hudson is an actual OS user with a shell and a home directory: /var/lib/hudson
/var/lib/hudson/jobs/{{job-name}}/workspace This is where your files go after a source check-out
The workspace ($WORKSPACE) directory is where you run unit tests before pushing via SSH.
Where are my files going?

Grant SSH user specific permissions
Allow the “deploy” group to write and own virtual host configuration files
sudo chgrp deploy /etc/apache2/sites-available /etc/apache2/sites-enabled
sudo chmod g+w /etc/apache2/sites-available /etc/apache2/sites-enabled
sudo chmod g+s /etc/apache2/sites-available /etc/apache2/sites-enabled
EDIT THE "/etc/sudoers" file
sudo visudo NOTE: Ubuntu may default to nano as editor; however, you can change it to vim or something else with the following command: sudo update-alternatives --config editor
ADD THE FOLLOWING LINE (under the "User privilege specification" section)
api ALL=(root) NOPASSWD: /usr/sbin/service apache2 *

Create the releases directory
NOTE: Login as “hudson” user, then SSH to “api” user.
sudo su – hudson
vim ~/.ssh/config Host 127.0.0.1 User api IdentityFile /var/lib/hudson/.ssh/api.pem Port 22 Protocol 2
ssh -i ~/.ssh/api.pem api@127.0.0.1
mkdir -p ~/apps/api.example.com/releases ~/apps/api.example.com/shared NOTE: RSA key fingerprint must be accepted once manually in order to automate authentication.

Configure Hudson for Git & Rsync
In Shell
git config --global user.name "Hudson”
git config --global user.email "hudson@example.com"
Hudson Job Configuration
“ example.com.api” >> “Configure” >> “Source Code Management” >> “Git”
URL of Repository: /media/psf/example.com.api NOTE: I cheated by linking a folder to my VM
“ Build” >> “Add Build Step” >> “Execute Shell” >> “Command”
rsync -avzH -e ssh --progress $WORKSPACE/ api@127.0.0.1:~/apps/api/releases/$BUILD_ID

Configure Build
Hudson Job Configuration (cont.)
“ Build” >> “Add Build Step” >> “Execute Shell” >> “Command”
ssh api@127.0.0.1 <<ssh-session
########## BEGIN: ssh commands ##########
cp ~/apps/api.example.com/releases/${BUILD_ID}/ops/etc/apache2/sites-available/example.com.api.vhost /etc/apache2/sites-available/
ln -nfs /etc/apache2/sites-available/example.com.api.vhost /etc/apache2/sites-enabled/
ln -nfs ~/apps/api.example.com/releases/${BUILD_ID} ~/apps/api.example.com/current
sudo /usr/sbin/service apache2 restart
########## END: ssh commands ##########
ssh-session
Now, run build, then browse to http://api.example.com/

A Restful web service API maps HTTP to code
Responds to common HTTP verbs/methods (GET, POST, PUT, DELETE)
Exposes a restful (not RPC, not Query) API
Restful: /users/1
RPC: /users/show/1
Query: /users/show?userId=1
Responds with a JSON content type (application/json)
Let’s build something more interesting to deploy

What is a Router class?
The “Router” class will allow us to match information of $_SERVER[‘REQUEST_URI’] to a controller class.
The “Router” class will allow us to match information of $_SERVER[‘REQUEST_URI’] to an anonymous function.
The “Router” class will allow us to match information of $_SERVER[‘REQUEST_URI’] to a global function.
Building the routing class

A Restful web service API
Support output buffering
Error & Exception Handling
Route optional segments
Parameter conditions
Content-type negotiation
Correctly respond to HEAD requests
Possible Future Enhancements

The Files
autoload.php.dist
bootstrap.php
phpunit.xml.dist
src/put/lib/RouterTest.php
src/lib/Router.php
src/www/index.php
Let’s code it together then deploy it

Bamboo
Team City
CruiseControl (PHPUnderControl)
Team Foundation Server
Hudson Alternatives

Books
Continuous Integration: Improving Software Quality and Reducing Risk Paul M. Duvall, Steve Matyas, Andrew Glover
Refactoring: improving the design of existing code Martin Fowler
Refactoring Databases : Evolutionary Database Design Scott W. Ambler, Pramodkumar J. Sadalage
Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation Jez Humble, David Farley
The Pragmatic Programmer: From Journeyman to Master Andrew Hunt, David Thomas
Agile Testing: A Practical Guide for Testers and Agile Teams Lisa Crispin, Janet Gregory

Reference
CI Feature Matrix http://confluence.public.thoughtworks.org/display/CC/CI+Feature+Matrix
Free Hosted Continuous Integration Environment http://www.fazend.com/
Why are you still not using Hudson? http://blog.uncommons.org/2008/05/09/why-are-you-still-not-using-hudson/
Continuous Integration (Martin Fowler) http://martinfowler.com/articles/continuousIntegration.html

http://frontrangephp.org	1188
http://www.frontrangephp.org	8
http://webcache.googleusercontent.com	5
http://translate.googleusercontent.com	2
http://static.slidesharecdn.com	1
http://frontrangephp.local	1
http://www.netvibes.com	1

Tame Your Build And Deployment Process With Hudson, PHPUnit, and SSH

by David Stockton on Jan 27, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

7 Embeds 1,206

Statistics

Tame Your Build And Deployment Process With Hudson, PHPUnit, and SSH — Presentation Transcript