Drupal Continuous Integration with Jenkins - Deploy

Drupal Continuous Integration With Jenkins – Deploy Classic Graphics – Charlotte, NC By: John W Smith License: Creative Commons CC0

Description / Purpose1. In the following slides, we will be describing simple Jenkins jobs that can be used to automate the deployment your git repository based code (not specific to Drupal, although that is our purpose) to your Demo, Live, Staging and/or Testing environments. First, we will configure a Jenkins job that has the simple task of deploying code to a single server / environment. Although this is not optimal, as it requires a separate Jenkins job for each site / server / environment, it is simple to setup and requires only basic scripting. Second, we will introduce a couple of scripts that will allow deploying multiple repositories to their corresponding servers based on parameters passed to the Jenkins job. This goes back to our original recommendation to name your repositories, servers, Jenkins jobs, etc using a common simple format.2. We assume you have already followed the previously released document “Drupal Continuous Integration with Jenkins” describing the Basic install, setup and configuration of Jenkins for Drupal Continuous Integration.

Disclaimer This document does NOT go into the complexities ofsecuring your system from internal or external attackers,please ensure your system is secure whether or not it isaccessible from the internet. Also, please consult with your organizations securityand system administrators before making any changes toexisting systems configuration or authentication models,and before introducing new systems into the developmentand / or production server environments.

System Requirements2GHz+ Multi-Core CPU4GB+ RAM250GB or more depending on the number of concurrent builds,and build history, artifacts, etc that you will keeping around forreview.Software needed: Jenkins 1.xxx (I used 1.433.x) Java JDK/JRE 1.x MySQL 5.x PHP 5.x Drush (latest version) Drupal 6.x Core SimpleTest patch file (included with SimpleTest Drupal Module)

References1. Points of Contact The Points of Contact (POC) for this documentation are: Technical Lead: David Norman (davidn@knowclassic.com) Project Lead: John Smith (johns@knowclassic.com) or (JSmith@i1Technologies.com)

Welcome!Jenkins is here to help. Specifically, Jenkins is an "extensible continuous integration (CI)server". From a 10,000ft view, Jenkins is a tool for offloading the deployment andautomated testing of a software application. You write your code, push it into versioncontrol, and Jenkins will take over the task of grabbing that code, running an installation /deployment process, testing the application (if you provide it with a test environment),and reporting back to you those test results. ~ (http://thinkshout.com/blog/2010/09/sean/beginners-guide-using-hudson-continuous-integration-drupal)There are a number of different CI tools out there, but we (Classic Graphics) have chosenJenkins, as it seems to be the leader in the Drupal world. Jenkins, for us, provides a web-based user interface, easy integration via LDAP to our Windows AD Domain, a friendlyway to run scripts, integration with CVS / Subversion / Git repositories, and of course itworks seamlessly with multiple projects / builds simultaneously. Also, as an added bonus,Jenkins has a Debian repository that we can use to install it on our Ubuntu serverswithout worrying about clobbering other libraries and avoid any dependency nightmares.

Create a Server Specific Deployment Job Here, we will be creating a simple server specific deployment job for our Drupal environment, this job could then be used as the base for other deployment sites / servers. Since every shops environment, testing and verification requirements will differ, we leave the advanced deployment scripting to the user of this document. The Jenkins job setup in this tutorial makes a few assumptions: 1. You have setup pub-key based auth over ssh to your git repository and possibly the destination web servers, depending on your configuration this type of setup may or may not work for you. 2. You have Drupal core, all required modules, and all custom modules located within your git repository. However, the script could be easily modified to retrieve the latest released version of Drupal and any other required modules using Drush if this is not the case in your organization. 3. Access to deployment account private and public keys (if required) and / or the accounts password. Deployment account MUST have SSH access to the destination web server(s) AND write permission to each sites “DocumentRoot” directory. Obviously this type of setup has security implications, please consult with your systems security officer / administrator.

1. First, open up your favorite browser and point it to the Jenkins server (http://jenkins.ourdomain.com:8080). Click the “log in” link in the upper right-hand corner of the browser window.2. Enter your login credentials and click “log in”.

3. For this job, we will need an additional plugin installed and enabled. Click the “Manage Jenkins” link on the left.4. Click the “Manage Plugins” link.5. Scroll down to find the “SSH Plugin”, click the check box to enable it. Scroll to the bottom of the page and click “Install without restart”.6. Click the “Return to Dashboard” link in the left menu to return to the Jenkins Dashboard once the plugin has been installed.

7. Click the “Manage Jenkins” link in the menu.8. Click the “Configure System” link.9. Scroll down to the SSH section, and add the appropriate information for your destination server. We have the SSH Server advanced view shown in this screen shot which allows entry of server login / password information for the SSH account to be used.10. Add as many entries as needed. You will need one for each server you plan to use as a destination.11. Click “Save” when done.

12. Click the “New Job” link in the menu. Enter “Deployment Test” and select “Build a free-style software project”. Click “Ok” when done. NOTE: When creating these jobs, in order to simplify the creation of the “git” post-receive bash script to launch a build remotely, you should consider naming your jobs the same as the name of your repository/repo-directory name. So, if you have your repos on your git server in the /var/git/reponame.git directory,then this job would be called reponame. As we progress through this tutorial, it will become apparent why this makes things easier.

13. Now let’s configure this simple job.14. Scroll down to the Build Environment and we’ll enable the “Send files or execute commands over SSH after the build runs” option.15. Select the server you will be deploying to from the Drop down list.16. Enter the following text in the “Exec Command” text area without the quotes. “git-update.sh user.account branch” where: • git-update.sh – is a script defined later in this document • user.account – is the user the script will use to retrieve the git repository code (should be the same account as the SSH account in a perfect world) • branch – the repository branch you want to deploy with this script (most likely master in all instances except the test server environment)17. Enable the “Exec in pty” option.18. Click “Save”.

Create a Generic Deployment Job Here, we will be creating a generic deployment job for our deployment environment. Since every shops environment, testing and verification requirements will differ, we leave the advanced deployment scripting to the user of this document. The Jenkins job setup in this tutorial makes a few assumptions: 1. You have setup pub-key based auth over ssh to your git repository and the destination web servers, depending on your configuration this type of setup may or may not work for you. Alternately, you could use an expect script to enter the login credentials for you, however this is somewhat insecure as the password would be available in the expect script. 2. You have code (Drupal core, all required modules, and all custom modules) located within your git repository. However, the script could be easily modified to retrieve the latest released version of Drupal and any other required modules using Drush if this is not the case in your organization. 3. Access to the deployment account private and public keys (if required) and / or the accounts password. Deployment account MUST have SSH access to the destination web server(s) AND write permission to each sites “DocumentRoot” directory. Obviously this type of setup has security implications, please consult with your systems security officer / administrator.

1. First, open up your favorite browser and point it to the Jenkins server (http://jenkins.ourdomain.com:8080). Click the “log in” link in the upper right-hand corner of the browser window.2. Enter your login credentials and click “log in”.

3. Click the “New Job” link in the menu. Enter “Deployment Test” and select “Build a free-style software project”. Click “Ok” when done. NOTE: When creating these jobs, in order to simplify the creation of the “git” post-receive bash script to launch a build remotely, you should consider naming your jobs the same as the name of your repository/repo-directory name. So, if you have your repos on your git server in the /var/git/reponame.git directory,then this job would be called reponame. As we progress through this tutorial, it will become apparent why this makes things easier.

4. Now we need to fill in some basic configuration for our newly created Deploy-Test job as shown in the following screenshots. a. Enter a description for our job (not required) b. If you would like to setup automatic deletion of old builds, check the “Discard Old Builds” checkbox (not needed for this build, as nothing will be downloaded or created in this deployment scenario). c. You will probably want to leave project based security enabled, add any additional users and set permissions as required.

d. We will configure 3 build parameters for this build.e. The first parameter is “REPO” and will be used to select the git repository to deploy.f. The second parameter is “BRANCH” and will determine the branch of the given repository to deploy. The “BRANCH” parameter should default to master in most cases.g. The third parameter is “SRVENV” and will contain the server enviroment we want to deploy the git repository to (in our shop we use [test, demo, staging, and live environments] ).

h. Check the “Execute concurrent builds if necessary” option.i. No Source Code Management needed for deployment jobs.j. Check “Trigger Builds remotely” option, and enter a token that can be used to deploy the sites. For security reasons, only persons designated to deploy code to the production environment should be given this token.k. Check the “Color ANSI Console Output” option. (not required)

#!/bin/bash -x ###### ## This script is an example ONLY and should be customized for the site/modules you are working on... ###### ###### # This is pretty straight forward actually, and can be accomplished a couple of different ways. This # script is designed to be used inside the Jenkins environment, with a job that can deploy a repository # to a matching server environment. # # Requirements: # SSH Keys setup for the Jenkins user account # SSH authorized_keys on destination servers must contain the jenkins account pub key in the # destination servers deployment account # The generic version of git-update.sh script included with this job script.l. Now we’ll add a BASH script to our job, # The deployment account on each destination server needs to have SSH keys setup to allow read- # only access to the git repository server. This will enable the deployment account to retrieve the shown to the right here. This will get # appropriate repository and branch to deploy without the need for human interaction. # you started, and can be expanded # Jenkins job parameters: # REPO = GIT Repository to deploy upon depending on your # BRANCH = GIT Repostiry branch to deploy # SRVENV = Should be one of the following for CGs environment / configuration. Your setup may requirements. # differ. # test - test server environmentm. That’s it for Jenkins using this # # staging - staging environment for final QA demo - demonstration server environment deployment method. Scroll down to # ###### live - production server environment the bottom of the job creation screen ### Configuration Variables ### and click “Save”. # Your deployment user account DEPLOY_ACCOUNT="deploy_user"n. We’ll now move on to the server side # The base domain for all your servers in your test/production environment this could also be moved # up to the Jenkins job parameters section if you need to deploy to several different domains script that will be used with both of DOMAIN="exampledomain.com" the above Jenkins deployment jobs. # set the deployment server based on supplied information if [ "x"${SRVENV%live} = "x" ]; then # deploying to the live / production environment DEPLOY_SERVER="${REPO,,*}.${DOMAIN,,*}" else # deploying to an environment other than live / production DEPLOY_SERVER="${REPO,,*}-${SRVENV,,*}.${DOMAIN,,*}" fi # now well simply execute the git-update.sh script (installed on the deployment # destination server) with the appropriate parameters ssh ${DEPLOY_ACCOUNT}@${DEPLOY_SERVER} git-update.sh ${DEPLOY_ACCOUNT} ${BRANCH}

Generic git Deployment Script #!/bin/bash script=`basename $0`; usage=$(cat <<EOF_USAGE USAGE: $script <cgnum> [branch] EOF_USAGE); if [[ $1 == ]]; then echo $usage >&2; exit 1; fi1. The following script will need to gitconfig=$(cat <<EOF_USAGE You need to configure your git identity and try again.n be placed on each server that will This information is required once for identifying who isn responsible for each tag this script creates during a deploy.n be used as a destination for one of n Example:n our Jenkins deployment jobs, and $ git config --global user.name “Git User"n $ git config --global user.email “guser@ourdomain.com"n should be placed within the path n Templates to copy/paste:n n and accessible / executable by the git config --global user.namen git config --global user.emailn deployment account. EOF_USAGE); if [ ! -e "$HOME/.git/config" ]; then echo -e $gitconfig >&2; exit 1; Fi HOSTNAME=`hostname` if [ ${HOSTNAME:(-8)} == "-staging" ]; then REPO=${HOSTNAME%-staging} ENVIRON="staging“ elif [ ${HOSTNAME:(-5)} == "-test" ]; then REPO=${HOSTNAME%-test} ENVIRON="test"

elif [ ${HOSTNAME:(-5)} == "-demo" ]; then REPO=${HOSTNAME%-demo} ENVIRON="demo“else REPO=${HOSTNAME} ENVIRON="live“fiif [[ $2 == ]]; then BRANCH=“master”else BRANCH=$2fiif [ -d "/tmp/clone/" ]; then echo "Removing clone directory...“ rm -rf "/tmp/clone/“fiecho "Cloning $BRANCH...“mkdir "/tmp/clone/“git clone -b "$BRANCH" ssh://$1@code.cgraphics.com/var/git/$REPO.git/ /tmp/clone/cd "/tmp/clone/“TAG=`date +%Y-%m-%d-%H-%M`HASH=`git log "$BRANCH" -n 1 --pretty=format:%h`echo "Tagging local clone...“git tag -a -m "$BRANCH $HASH to $ENVIRON by $1" "$ENVIRON-$TAG“echo "Pushing tag to origin...“git push origin "$ENVIRON-$TAG“echo "Syncing the $BRANCH clone to public_html...“rsync -rl --exclude=private --exclude=".git" --exclude=".gitignore" /tmp/clone/ /var/www/echo "Removing clone directory...“rm -rf /tmp/clone/echo "Launch complete!“echo "Template for pasting to a related issue:“echo "Deployed $HASH with tag $ENVIRON-$TAG."

2. That’s it, just place this script in the path, setup one of the previous Jenkins jobs modified appropriately for your environment, and you will be on your way to automated deployment of your code via your Jenkins CI Server.

Drupal Continuous Integration with Jenkins - DeployEnjoy, and feel free to contact us with any suggestions,corrections or mistakes we may have made or thatinadvertently slipped between the cracks.John W Smith (johns@knowclassic.com) or (Jsmith@i1Technologies.com)

http://www.gamedevportal.com	16
http://support.diplux.com	4
http://paper.li	1

Drupal Continuous Integration with Jenkins - Deploy

by John Smith on Dec 30, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 21

Statistics

Drupal Continuous Integration with Jenkins - Deploy — Presentation Transcript