Drupal Continuous Integration with Jenkins - The Basics

Drupal Continuous Integration With Jenkins Classic Graphics – Charlotte, NC By: John W Smith License: Creative Commons CC0

Description / Purpose1. In the following slides, we will build an integrated Jenkins CI environment that can be used for testing of the Drupal CMS system and modules.2. Since we have not fully integrated Jenkins into our development environment, we’ll explain here our current implementation and will continue to add to this document as we improve upon it in the future.3. The purpose of this document is to provide the procedures and criteria to install, configure and verify Drupal code commits via a Jenkins Continuous Integration Server environment. Depending on the complexity of your environment, and the amount of integration needed for existing infrastructure, you should be able to have a basic Jenkins system capable of running Drupal automated testing within a few hours.

Current Setup1. As of now, we have a Jenkins server setup that performs a “Build” in response to a “Push” to our Git repository.2. Immediately following the “Push”, our post-receive script accesses a URL on the Jenkins server, passing it the name of the Git repo, the branch that was updated and the Key/Token for authentication.3. With this information, Jenkins clones the repository and then kicks off a BASH script that switches the cloned repo to the “commit” branch and then utilizes Drush to install a new site and perform automated SimpleTest testing of the committed code.

Future Plans1. Currently, we have plans to email our developers if any of their committed code causes a build to fail, email successes to the deployment team, etc.2. Integration with a headless Selenium Web Based testing server setup.The percentage of our current test coverage is very limited atbest. This is a far cry from test-driven development and totalContinuous Integration, but its a start.

Disclaimer This document does NOT go into the complexities ofsecuring your system from internal or external attackers,please ensure your system is secure whether or not it isaccessible from the internet. Also, please consult with your organizations securityand system administrators before making any changes toexisting systems configuration or authentication models,and before introducing new systems into development and/ or production server environments.

System Requirements 2GHz+ Multi-Core CPU 4GB+ RAM 250GB or more depending on the number of concurrent builds, build history, artifacts, etc that you will be wanting to keep around for archival / review purposes. Software needed: Jenkins 1.xxx (I used 1.433.x) Java JDK/JRE 1.x MySQL 5.x PHP 5.x Drush (latest version) Drupal 6.x Core SimpleTest patch file (included with SimpleTest Drupal Module)

References1. The following references have direct applicability to the system. Ubuntu Server Community Documentation. (https://help.ubuntu.com/10.04/serverguide/C/index.html) Jenkins Continuous Server Documentation. (https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins) Getting Hudson (Jenkins) Up and Running with Drupal (http://thinkshout.com/blog/2010/09/sean/beginners-guide-using-hudson- continuous-integration-drupal)2. Points of Contact The Points of Contact (POC) for this documentation are:  Technical Lead: David Norman (davidn@knowclassic.com)  Project Lead: John W Smith (johns@knowclassic.com) or (JSmith@i1Technologies.com)

ScopeThis document will cover the installation of a basicUbuntu 10.04 LTS Server and other software associatedwith running a Jenkins CI environment capable oftesting Drupal source code commits. We assume forthis tutorial / documents purpose, that the server isrunning a local MySQL Server and ClamAV daemons.

Welcome!Jenkins is here to help. Specifically, Jenkins is an "extensible continuous integration (CI)server". From a 10,000ft view, Jenkins is a tool for offloading the deployment andautomated testing of a software application. You write your code, push it into versioncontrol, and Jenkins will take over the task of grabbing that code, running an installation /deployment process, testing the application (if you provide it with a test environment),and reporting back to you those test results. ~ (http://thinkshout.com/blog/2010/09/sean/beginners-guide-using-hudson-continuous-integration-drupal)There are a number of different CI tools out there, but we (Classic Graphics) have chosenJenkins as it seems to be the leader in the Drupal world. Jenkins, for us, provides a web-based user interface, easy integration via LDAP to our Windows AD Domain, a friendlyway to run scripts, integration with CVS / Subversion / Git repositories, and of course itworks seamlessly with multiple projects / builds simultaneously. Also, as an added bonus,Jenkins has a Debian repository that we can use to install it on our Ubuntu serverswithout worrying about clobbering other libraries and avoid any dependency nightmares.

Installing Ubuntu 10.04 LTS1. Insert the Ubuntu Server CD into your system and boot from it. Select your language:

2. Select “Install Ubuntu Server” from the following screen:

3. Choose your installation language again.

4. Select the location of your server installation.

5. Choose a keyboard layout (if you select “Yes” here to detect keyboard layout, the installation will ask you topress a series of keys and try to detect your keyboard layout).

6. The installer will now check your hardware and configure the network withDHCP if there is a DHCP server on the network.

7. Now we’ll need to enter a hostname for the server.

8. Setup will now attempt to detect your timezone, if it successfully detects thecorrect timezone, select “Yes” to continue. Otherwise, select “No” to manuallyselect the appropriate timezone from the list.

9. Your hard disk drive will now be detected, and the partition setup wizard will belaunched. Select “Guided – use entire disk and set up LVM” or the typerecommended by your IT department.

Select disk:

Write partition information to disk before creating LVM setup:

Size to use for LVM group.

Confirm writing changes to disk.

10. The installation will now install the base Ubuntu 10.04 system.

11. Once the base system has been installed, you will need to create a standard useraccount for the system. Enter the users Full Name at the prompt.

Next you’ll need to enter the user login name, a suggested name will be supplied.

Next you will be asked for a password for this user.

Select whether or not to encrypt your home directory based your organizations ITpolicy and procedures.

12. Setup Package Manager proxy settings.

Setup will now configure “apt” and download the appropriate files and repositorydata files.

Next you will be asked how you would like to handle updates, select the optionbased on your organizations IT policy and procedures.

13. Next select the following 2 software groups to install on your new server.

15. The selected software will be downloaded and installed on the server system.

16. When asked if you would like to install the GRUB boot loader on your system,select “Yes”.

17. We are now finished with the basic Ubuntu 10.04 system install.

System Configuration1. Login to your system using the account we created during the installation. Once logged in, we will use the “sudo” command to switch to the root user. user@jenkins:~$ sudo su - [sudo] password for user: root@jenkins:~#2. Edit the network configuration to assign a static IP address instead of obtaining an DHCP assigned address. We will be modifying the “eth0” portion of this file so that it looks like the following (substituting your network specific IP settings).

root@jenkins:~# vim /etc/network/interfaces We will be modifying the “eth0” portion of this file so that it looks like the following (substituting your network specific IP settings).# The primary network interfaceauto eth0iface eth0 inet static address 192.168.0.100 netmask 255.255.255.0 network 192.168.0.0 broadcast 192.168.0.255 gateway 192.168.0.1

Restart the network sub-system.root@jenkins:~# /etc/init.d/networking restartNext we need to edit the “hosts” file, remember to use your networkspecific IP address.root@jenkins:~# vim /etc/hosts127.0.0.1 localhost.localdomain localhost 192.168.0.100 jenkins.ourdomain.comjenkins# The following lines are desirable for IPv6 capable hosts::1 localhost ip6-localhost ip6-loopbackfe00::0 ip6-localnetff00::0 ip6-mcastprefixff02::1 ip6-allnodesff02::2 ip6-allroutersSetup the “hostname” file.root@jenkins:~# echo jenkins.ourdomain.com > /etc/hostnameroot@jenkins:~# /etc/init.d/hostname restart

Additional SoftwareHere we will install additional software needed to be installed before weconfigure Jenkins. We’ll take care of the rest of the software configurationafter Jenkins is up and running.1. Just to cover the most popular and supported repository software, we’ll install CVS, Subversion and Git. We will also install the ClamAV daemon to allow Jenkins to scan our project artifacts for virus signatures. a. Install the software packages. root@jenkins:~# aptitude install git-core subversion cvs clamav-daemon ant openjdk-6-jdk b. Configure clamd for TCP operation. root@jenkins:~# vim /etc/clamav/clamd.conf c. We need to add the following 2 lines to the configuration file. TCPSocket 3310 TCPAddr 127.0.0.1

2. For our purpose in the use of Jenkins for Drupal testing, we will also need to install Drush. The version available through the Ubuntu 10.04 repo is quite old however, so we will first download the latest released version (7.x-4.5) from http://drupal.org/project/drush (http://ftp.drupal.org/files/projects/drush-7.x-4.5.tar.gz). Follow the instructions in the included INSTALL.txt to install, my personal preference is to extract the tarball into /usr/share (or /usr/local/share) and then symlink /usr/share/drush/drush (/usr/local/share/drush/drush) to /usr/bin/drush (/usr/local/bin/drush), just to make things simple. root@jenkins:~# wget http://ftp.drupal.org/files/projects/drush-7.x-4.5.tar.gz root@jenkins:~# tar –C /usr/share –xzf drush-7.x-4.5.tar.gz root@jenkins:~# ln –s /usr/share/drush/drush /usr/bin

3. Now we’ll update the system to download any security updates released since our installation CD was created. After the upgraded packages have been installed, we will need to reboot the system before proceeding. root@jenkins:~# aptitude update root@jenkins:~# aptitude full-upgrade -root@jenkins:~# shutdown –r now4. All done, the system should be ready for installing Jenkins when it comes back up.

Install JenkinsJenkins can be easily installed on Debian based systems using therepository provided by the kind folks over at jenkins-ci.org. Simply followthese instructions to add the Jenkins repository to your system and installJenkins along with its dependencies.1. Setup Jenkins CI Repository. a. First we will need to add the repository signing key from jenkins-ci to our key chain. root@jenkins:~# wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add - b. Next we create the jenkins.list repo file for jenkins-ci.org. root@jenkins:~# echo “deb http://pkg.jenkins-ci.org/debian binary/” > /etc/apt/sources.list.d/jenkins.list

c. Now update the repository data. root@jenkins:~# aptitude update2. Install Jenkins and its dependencies using aptitude. root@jenkins:~# aptitude install jenkins3. That’s it for getting Jenkins installed, the rest of the configuration happens from the web interface.

Configure Jenkins Server1. Point your favorite web browser (FireFox of course) to http://{ip of server}:8080 (or if you have DNS entries for the system already in place), http://jenkins.ourdomain.com:8080. You should now see the Jenkins Dashboard.

2. The first thing we’ll need to do is secure our Jenkins install. (using this guide, we’ll just enable Jenkins own internal user management for simplicity). a. Click the “Manage Jenkins” link on the left-hand side of the page.

b. Click the “Configure System” link in the right-hand content portion of the page.c. Scroll down until you see the “Enable Security” checkbox and check it. Then select “Jenkin’s own user database” under “Security Realm”, leave “Allow users to sign up” checked for now. Scroll to the bottom and click the “Save” button.

d. Add yourself as a user by clicking the “sign up” link now located in the upper right-hand corner of the webpage.

And then enter your user / logininformation on the form provided, andthen clicking the “Sign up” button.

e. Now we will modify the security configuration to disabled user account creation and provide a finer grained security model. Click “Manage Jenkins” -> “Configure System” again.f. Scroll down to “Security Realm” -> “Jenkin’s own user database” and remove the checkmark from “Allow users to sign up”.g. Scroll down and under “Authorization” select “Project-based Matrix Authorization Strategy”. Enter the username used above in the “User/group to add:” text box and click “Add”. Select the permissions for the Anonymous user and “All” permissions for the “Administrative” user we just created.h. We should now have a Jenkins install that will only allow anonymous users to see jobs that have been created by authorized users. Additional Users may be added to the system by going to “Manage Jenkins” -> “Manage Users”.

3. Next we need to apply any updates to Jenkins plugins that are available and restart Jenkins. Click “Manage Jenkins” -> “Manage Plugins”. If there are any available updates, select “All” and click “Install”.

Click the “Restart Jenkins when …”checkbox at the bottom of the page.

4. After the plugins have been updated, Jenkins will restart and you will be directed to the “Login” page. Enter your credentials and click “Login”.5. Click “Manage Jenkins” -> “Manage Plugins”. Click on the “Available” tab to display a list of modules available for install. We will be select the following modules (Audit Trail Plugin, Create Job Advanced Plugin, Blame Upstream Committers Plugin, Email-ext Plugin, Status Monitor Plugin, All Changes Plugin, Checkstyle Plugin, DRY Plugin, Log Parser Plugin, PMD Plugin, Violations, Workspace Cleanup Plugin, ClamAV Plugin, AnsiColor Plugin, Git Plugin) which may be listed in multiple locations, we only need to check it once on the page, not every location . Not all of these are required, but most should be useful for Drupal builds, feel free to leave some out or add additional modules as required. Once complete, click the “Install” button at the bottom of the page. Jenkins will restart when finished installing the modules, and you will be presented with the login screen.

6. After logging into the Jenkins system again, click “Manage Jenkins” -> “Configure System”. a. Scroll down to the “Create Job Advanced” section. We want to check the first 3 checkboxes.

b. Next we will add the path to our Java JDK installation. Enter a name for the install, then uncheck the “Install Automatically” and enter the correct path for JAVA_HOME (should point to /usr/lib/jvm/java-6- openjdk/ on Ubuntu systems using the standard repository packages).

c. Next we want to make sure the Git: configuration for the source code management programs is correct, or enter the correct values as shown below.

Git Username / email:CVS/Subversion:

d. Next we need to configure the location of our Apache Ant installation. Enter a name for the install, and then uncheck the “Install Automatically” checkbox, then enter the correct path to our Apache Ant installation (/usr/share/ant if using standard Ubuntu repository).e. Set /bin/bash as our shell.

f. Configure Jenkins email settings.g. Configure the ClamAV scanner and then click the “Save” button.

That’s it for the basic configuration of Jenkins. Later on we’ll use the web interfaceto configure a sample job that will perform the following tasks: Download source code from the repository Use drush to setup / install the Drupal instance Enable the Coder module to perform Drupal Style analysis of the given modules Enable the SimpleTest module and run available SimpleTests on the module(s) / Module Group(s) Run PHP Mess Detector against internally developed modules Run PHP Copy and Paste Detector against internally developed modules Collect XML output from above tests and create trend graphs to track internal modules Notify developers when committed code breaks build Scan source code for viruses

System ConfigurationNext we’ll be modifying the default configuration of Apache and MySQL. Since we won’t beusing any live data for these tests, standard configuration is find for the most part.1. Apache a. We need to install PHP5 for both apache and command line use. At a minimum, the modules included in the command below should also be installed. You may install other PHP5 modules if your drupal installation requires them to be installed. root@jenkins:~# aptitude install php5 php5-cli php5-gd php5-curl php5-dev php-pear NOTICE: If you installed either the php5-imagick or php5-mcrypt modules above, you will need to edit their configuration files to avoid errors when running SimpleTest tests. Comments beginning with ‘#’in configuration files have been deprecated, all comments should start with the ‘;’ character. (See Ubuntu bugs 573436 and 556469)

i. Edit the /etc/php5/cli/php.ini configuration file as follows root@jenkins:~# vim /etc/php5/cli/php.ini Set the following configuration variables, everything else can remain as is in the configuration file. Memory limit for CLI environment has been set to unlimited (-1). short_open_tag = Off max_execution_time = 600 memory_limit = -1ii. Make the same edits to /etc/php5/apache2/php.ini, with the exception of the memory_limit variable which should be set to a minimum of 128MB. memory_limit = 128M

b. We need to enable the rewrite module. root@jenkins:~# a2enmod rewritec. Next we’ll need to modify the standard default site configuration to allow overrides. root@jenkins:~# cd /etc/apache2/sites-available/ root@jenkins:/etc/apache2/sites-available# vim default The finished file should look like the following: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride All

Order allow,deny allow from all </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice,warn, error, crit, # alert, emerg. LogLevel info CustomLog /var/log/apache2/access.log combined Alias /doc/ "/usr/share/doc/“ <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None

Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 ::1/128 </Directory> </VirtualHost> d. Now we need to restart apache so the configuration changes will be applied. root@jenkins:~# /etc/init.d/apache2 restart2. MySQL a. Create a MySQL user to use for testing. root@jenkins:~# mysql -u root –p mysql Enter password: mysql> CREATE USER testuser@localhost IDENTIFIED BY testuser; mysql> GRANT ALL ON *.* TO testuser@localhost; mysql> quit; b. That should do it for MySQL.

3. Now we will install some additional PHP tools. a. First let’s upgrade pear to the latest version. root@jenkins:~# pear channel-update pear.php.net root@jenkins:~# pear upgrade-all b. PHP Mess Detector (phpmd) i. First we need to add the required pear channels. root@jenkins:~# pear channel-discover pear.phpmd.org root@jenkins:~# pear channel-discover pear.pdepend.org ii. Install some dependencies for phpmd / pdepend with aptitude. root@jenkins:~# aptitude install imagemagick libmagickwand-dev iii. Now we can install phpmd. root@jenkins:~# pear install --alldeps phpmd/PHP_PMD root@jenkins:~# if [ ! -e /etc/php5/conf.d/imagick.ini ]  then  echo extension=imagick.so > /etc/php5/conf.d/imagick.ini  fi root@jenkins:~#

iv. This following step is optional. However, if you want to integrate phpmd with ant to check your builds, you’ll need to do the following additional step. We need to download and install the ant- phpmd jar file, and place it into the ant lib directory. root@jenkins:~# wget http://static.phpmd.org/ant-phpmd/0.1.2/ant-phpmd.jar -O /usr/share/java/ant-phpmd-0.1.2.jar root@jenkins:~# ln -s /usr/share/java/ant-phpmd-0.1.2.jar /usr/share/java/ant-phpmd.jar c. PHP Copy Paste Detector i. First we need to add the required pear channels. root@jenkins:~# pear channel-discover pear.phpunit.de root@jenkins:~# pear channel-discover components.ez.no ii. Now we can install phpcpd. root@jenkins:~# pear install --alldeps phpunit/phpcpd4. We’re done…. Simple right!?

Create a Drupal Job in Jenkins Here, we will be creating a simple / basic job to test Drupal, and any custommodules just to make sure everything is in working order. Since every environmentand required testing is different, we will leave the advanced job creation anddeployment scripting to the end-user of this document.The Jenkins job setup in this tutorial makes a couple of assumptions: 1. You have setup pub-key based auth over ssh to your git repo, depending on your configuration this type of setup may or may not work for you. 2. You have Drupal core, all required modules, and all custom modules located within your git repository. However, the script could be easily modified to retrieve the latest released version of Drupal and any other required modules using Drush if this is not the case for you organization.

1. Open up our favorite web browser and point it to our new Jenkins server (http://jenkins.ourdomain.com:8080). Click the “log in” link in the upper right-hand corner of the browser window.

2. Enter your login credentials and click “log in”.

3. Click the “New Job” link in the menu. Enter “Drupal Test” and select “Build a free-style software project”. Click “Ok” when done. NOTE: When creating these jobs, in order to simplify the creation of the “git” post-receive bash script to launch a build remotely, you should consider naming your jobs the same as the name of your repository/repo-directory name. So, if you have your repos on your git server in the /var/git/reponame.git directory,then this job would be called reponame. When viewing the post- receive-jenkins script below, it will become apparent why this makes things easier.

4. Now we need to fill in some basic configuration for our newly created Drupal-Test job as shown in the following screenshots. a. Enter a description for our job (not required) b. If you would like to setup automatic deletion of old builds, check the “Discard Old Builds” checkbox. Additional options will be displayed. c. You will probably want to leave project based security enabled, add any additional users and set permissions as required. d. Next check the box labeled “This build is parameterized” and add parameters as needed. (optional) I have added a parameter for branch below, this will allow you to include the parameter in a URL from a post-commit to build a specific branch of the code, or leave it out to build the master branch. e. We’ll probably also want to be able to have concurrent builds going at the same time, just in case you have multiple developers that could be submitting changes at the same time.

f. Next we need to setup our repository information. As stated earlier, I have setup pub-key auth for accessing our remote repository.g. At some point you’ll want to be able to trigger builds remotely, so we’ll check the “Trigger builds remotely” option, and fill in the token field with a string used to uniquely identify and verify the build. This ensures that only people knowing the correct “Token” are able to kickoff a build remotely.h. Check the “Delete workspace before build starts” to ensure you are starting from a clean slate with each build.

i. Check the “Color ANSI Console Output” to capture any color output from your scripts.j. Next we need to create a build script. Scroll down to the “Build” section, and click the “Add build step” and select “Execute Shell” from the dropdown menu.

#!/bin/sh -x # First we need to create an empty DB for drupal using our testuser account mysqladmin -u testuser -ptestuser create ${BUILD_TAG} # Change to our working directory cd ${WORKSPACE} # Switch to the specified repository branch #(this is where our custom BRANCH parameter comes into play) git checkout ${BRANCH} # apply the drupal core patch for drupal 6.x # assumes you have the simpletest module installed already patch -p0 < sites/all/modules/simpletest/D6-core-simpletest.patch # use drush to install a basic site with defaults enabled using our empty db drush si --yes --db-url="mysql://testuser:testuser@localhost/${BUILD_TAG}"k. You will then be able to add a script (BASH) # just check the status to make sure we are good in the provided textbox to execute our drush --uri=http://customsite.example.com/ status Drupal build tests. I have included a sample # Change to our multi-site directory and link the default settings file here cd sites/customsite.example.com script here that runs PHP Mess Detector and ln -sf ../default/settings.php PHP Copy and Paste Detector on our custom # go back to the working directory so Drush can find things cd ${WORKSPACE} Drupal modules, the Coder module to check # create the directory to put our logs in our code against Drupal coding standards mkdir logs and SimpleTest(s) where available. We’ll # enable the coder module and run the coder based tests then disable it drush --yes --uri=http://customsite.example.com/ pm-enable coder,simpletest pipe the output of these checks into a # enable simpletest and other (required) modules that we will be testing. “logs” directory that we create in the script, drush --yes --uri=http://customsite.example.com/ pm-enable simpletest,{comma separated list of modules to enable, not including the standard/required Drupal modules} and use Jenkins plugins PMD, Checkstyle and DRY to parse and track the build history. # Run coder against our custom modules for MOD in {space separated list of modules to run coder against}; do drush --uri=http://customsite.example.com/ coder checkstyle ${MOD} > logs/coder- ${MOD}.xml 2>/dev/null done # Run the SimpleTest tests for our Custom module for GROUP in {space separated list of module(s) and/or module group(s) to test}; do drush --uri=http://customsite.example.com/ test-run $GROUP} 2>/dev/null done # Lets run the PHP Mess Detector on our custom modules phpmd sites/customesite.example.com/ xml codesize,unusedcode --reportfile logs/phpmd- drupal.xml 2>/dev/null # Run the PHP Copy Paste Detector on our code phpcpd --log-pmd logs/phpcpd-drupal.xml sites/customsite.example.com 2>/dev/null # drop this build database mysqladmin -f -u testuser -ptestuser drop ${BUILD_TAG}

l. Under “Post Build Actions” we will select “Publish Checkstyle analysis results” and enter the “Checkstyle results” as “logs/coder-*.xml” (or a pattern matching your script if different).m. Select “Publish PMD analysis results” and enter the “PMD results” as “logs/phpmd-drupal.xml” (or a pattern matching your script if different).n. Select “Publish duplicate code analysis results” and enter the “Duplicate code results” as “logs/phpcpd-drupal.xml” (or a pattern matching your script if different).o. We want to save our artifacts (code snapshot). So check the “Archive the artifacts” checkbox.p. Yeah, it’s PHP code, but someone could have embedded something nasty in an image file, DB insert, etc. Let’s check it just to be on the safe side! Enable the ClamAV scanner by checking the “Check for viruses” option.

q. We want to parse the output from our script, and mark builds accordingly, so we’ll check “Console output parsing” and both sub-options “Mark build Unstable on Warning” and “Mark build Failed on Error”.r. Select “Record fingerprints of files to track usage” and “Fingerprint all archived artifacts”.s. Select “Email Notification” enter your team email address (or mailing list, etc). Optionally select one or both of the sub- options for email notification.t. And the last option we will select for this build is the “Status Monitor” option.u. To save the configuration, click on the “Save” button at the bottom of the page.

1. Now we’re ready for a test run, click “Build Now” on the left-hand side of the page.2. How did we do? Did it work? How did your dev team do? Any warnings or errors in the code?

Git Post-Receive Processing #!/bin/bash1. We need to create a post-receive script while read oldrev newrev refname; do for git (assuming git is being used) to break; done; remotely trigger the Jenkins job each # find the branch of the push time an update is pushed to the git BRANCH_NAME=${refname##refs/heads/} repository. This script should be placed # we also need to find what our repository name is, this in a directory that is accessible by all # script assume that all repositories are stored in git users (perms 755). This is an # /some/directory/tree/reponame.git if [ $(git rev-parse --is-bare-repository) = true ]; then example script that provides basic REPONAME=$(basename $PWD) else functionality. We’ll expand upon it a REPONAME=$(basename $(readlink -nf "$PWD"/..)) little later to provide useful services to fi # strip off the .git part if present our build process: REPONAME=${REPONAME%.git} # Now we just do a simple http request to the jenkins server wget -q -O - -t 1 "http://jenkins.ourdomain.com:8080/job/${REPONAME}/buildWithParameters?to ken=YOUR-TOKEN&BRANCH=${BRANCH_NAME}" > /dev/null

2. Next we will need to modify each of the existing repositories .git/hooks/post-receive script as follows: #!/bin/sh # # see contrib/hooks/ for a sample, or uncomment the next line and # rename the file to "post-receive". #. /usr/share/doc/git-core/contrib/hooks/post-receive-email # IF we uncomment the line above, since post-receive arguments are passed through # <STDIN>, the above script will pull those off <STDIN> and we have no way of # grabbing them. There are several solutions to this, the one I have selected below # serves our purpose nicely. # get the <STDIN> input into a variableINPUT=$(cat) # now we can just pipe those args to any number of scripts in the same fashion, # without having to rewrite them to accept command line arguments echo ${INPUT} | /usr/share/doc/git-core/contrib/hooks/post-receive-email # and of course our new jenkins script echo ${INPUT} | /usr/share/doc/git-core/contrib/hooks/post-receive-jenkins

3. Next we’ll modify our original simple script (Item #1 above) showing how the post-receive script could be used to perform different actions based on the type of action that was performed on the git repository. This could possibly be a different Jenkins job, or the same job with more parameters passed to modify how the job script functions, or what procedures it uses to accomplish the selected task(s). Our script below creates an archive of the repository, and moves it to an archival location. a. First we will need to determine the type of change happening b. Then we need to know what type of revision is going on c. Finally we add some logic to act on the 2 pieces of additional information we have gathered from our git repo and passed parameters. d. We can easily add more cases that we could act on, or do different things based on any predetermined conditions that we can meet or be extracted from the information contained within the git push/transaction. Our modified script follows on the next few slides.

#!/bin/bash## Copyright (c) 2011 Classic Graphics# Released under GPLv2# read variables / parameters from <STDIN>while read oldrev newrev refnamedo breakdone# determine the type of change submitted# borrowed / copied from the post-receive-email scriptif expr "$oldrev" : 0*$ >/dev/nullthen change_type="create"else if expr "$newrev" : 0*$ >/dev/null then change_type="delete" else change_type="update"

fifi# Next we need to determine the revision type we are dealing with and act accordingly# borrowed / copied from the post-receive-email scriptnewrev_type=$(git cat-file -t $newrev 2> /dev/null)oldrev_type=$(git cat-file -t "$oldrev" 2> /dev/null)case "$change_type" in create|update) rev="$newrev" rev_type="$newrev_type" ;; delete) rev="$oldrev" rev_type="$oldrev_type" ;; *) rev="" rev_type="“esac

# we also need to find what our repository name is, this script assumes that all# repositories are stored in /some/directory/tree/reponame.gitif [ $(git rev-parse --is-bare-repository) = true ]then REPONAME=$(basename $PWD)else REPONAME=$(basename $(readlink -nf "$PWD"/..))fi# strip off the .git part if presentREPONAME=${REPONAME%.git}# now that we know what, lets do something useful with itcase "$refname","$rev_type" in refs/tags/live-*,commit) # well create a tarball of the tag here, and then move it to long term storage where it # can be archived and or moved to an Escrow area. TAG_NAME=${refname##refs/tags/} TMP_DIR="/tmp/${TAG_NAME}" # create our temp directory for repo mkdir -p ${TMP_DIR}

# clone the repo into the temporary directorygit clone /var/git/${REPONAME}.git ${TMP_DIR} 2>/dev/null 1>/dev/null# change into the tmp directorycd ${TMP_DIR}# checkout the tag we are going to archivegit checkout tags/${TAG_NAME} 2>/dev/null 1>/dev/null# The following code assumes there is a directory for the archives# @ /var/git/archives and that the git user has permissions to# write/create files and directories there.# make sure our path for the generated archive existsmkdir -p /var/git/archives/${REPONAME}.git# now well create the archive of our tag in a safe locationtar cjf /var/git/archives/${REPONAME}.git/${TAG_NAME}.tbz --exclude-vcs . 2>/dev/null 1>/dev/null# remove our temporary directorycd

rm -rf ${TMP_DIR} ;; refs/heads/*,commit) # find the branch of the push BRANCH_NAME=${refname##refs/heads/} # Now we just do a simple http request to the Jenkins server wget -q -O - -t 1"http://localhost:8080/job/${REPONAME}/buildWithParameters?token=DrupalTestBuild123456&BRANCH=${BRANCH_NAME}"> /dev/null ;; *) # anything else we simply ignore for now, other scenarios # and / or options can be added to the script as needed. ;;esac# exitexit 0

4. This could be extended further, to say deploy the live-{date} tagged items to the production server. Or possibly deploy tags like “staging-{date}” to a testing server (Part 2? / Deploy) or a Full Blown integrated testing environment that runs Selenium automated testing for QA Purposes (Part 3? / Testing). The possibilities are truly endless. I would also like to point out an excellent tutorial on getting Selinium + Jenkins setup on a headless linux system for running automated tests: http://centripetal.ca/blog/2011/02/07/getting-started-with-selenium-and-jenkins/Hopefully this has helped someone in the community in setting up there own Drupal CIenvironment, or at least given you a trotting start !Enjoy, and feel free to contact us with any suggestions, corrections or mistakes we may havemade or that inadvertently slipped between the cracks.John W Smith (johns@knowclassic.com) or (JSmith@i1Technologies.com)

http://www.linkedin.com	32
http://jimatic.blogspot.com	22
http://support.diplux.com	12
http://www.scoop.it	9
https://twitter.com	1

Drupal Continuous Integration with Jenkins - The Basics

by John Smith on Nov 01, 2011

Accessibility

Categories

Upload Details

Usage Rights

5 Embeds 76

Statistics

Drupal Continuous Integration with Jenkins - The Basics Presentation Transcript