Hudson: Your robotic butler


Published on

An introduction of how to use Hudson with Drupal from DrupalCamp NYC 8.

Published in: Technology
1 Comment
1 Like
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Hudson: Your robotic butler

  1. 1. Hudson Your Robotic Butler by @stevenmerrill Saturday, July 24, 2010
  2. 2. Hudson • Java-based continuous integration engine • Used to automate any regular job • Send notifications through a variety of channels Saturday, July 24, 2010
  3. 3. Why Use It? • Automate tasks that developers should run • Get detailed console logs for jobs without inserting into your Drupal database • Run jobs on any number of servers and chain them together contingent on the success • Notify users in a variety of media Saturday, July 24, 2010
  4. 4. Setting Up Hudson Cheerio! Saturday, July 24, 2010
  5. 5. Getting Hudson • Hudson is packaged as a .war file • Run it on a standard servlet container • Hudson labs also maintains a .deb file and an apt repository Saturday, July 24, 2010
  6. 6. Setup Tips • Run the Hudson server as a user with umask 002 and with the same primary group as your web server • This will make sure that permissions are correct for files when DrupalWebTestCase::drupalGet() or the like invoke your web server • Saturday, July 24, 2010
  7. 7. Setup, cont’d • You can also use Apache (or nginx, etc) to proxy Hudson so that you can run it without having to use :8080 in the URL • Use Apache’s mod_proxy and the instructions at Running+Hudson+behind+Apache Saturday, July 24, 2010
  8. 8. Configuring on Ubuntu • The simplest way to get up and running with Hudson is with Ubuntu and their apt repository • Saturday, July 24, 2010
  9. 9. Configuring on Ubuntu, cont’d • wget -q -O - | sudo apt-key add - • sudo sh -c "echo 'deb debian binary/' >> /etc/apt/sources.list" • sudo apt-get update • sudo apt-get install hudson • sudo make me a sandwich (this will take awhile) Saturday, July 24, 2010
  10. 10. Configuring on Ubuntu, cont’d • sudo usermod -G www-data hudson • sudo nano -w /etc/init.d/hudson • Add --umask=002 to DAEMON_ARGS • sudo service hudson stop • sudo service hudson start Saturday, July 24, 2010
  11. 11. You now have a robotic butler at your command. Saturday, July 24, 2010
  12. 12. Terminology • Every task you wish to automate using Hudson will is called a job • The directory on the file system where a job’s source files are checked out it the workspace • You create numerous build steps to execute whenever the job is run, which is called a build • You can take any number of files produced by the build and save them as artifacts Saturday, July 24, 2010
  13. 13. Terminology, cont’d • Each build has two status axes • Failure or success • Stable or unstable Saturday, July 24, 2010
  14. 14. This is Drizzle’s build farm at Saturday, July 24, 2010
  15. 15. Working with Jobs Credit: Saturday, July 24, 2010
  16. 16. Job Configuration • Each job has a unique name • Name your jobs like Drupal variables - lowercase and underscores • On the Ubuntu install, a folder will be created in /var/lib/hudson/jobs/[job name] . Saturday, July 24, 2010
  17. 17. Source Control Integration • Hudson can check source control for you and build your project when there are changes • Hudson will do a checkout and then grab any available changes and start a build once it has grabbed them • Other advantages include being able to have the notification plugins send out a list of changes and the ability to create Hudson users for each person who commits to your repository Saturday, July 24, 2010
  18. 18. Source Control, cont’d • For the job where you’ll be running your unit tests from, make sure you do not choose the option to do a clean checkout every time. • There are source control integration plugins for nearly every VCS out there, including the open- source standards cvs, svn, hg, bzr and git, and proprietary ones such as Perforce, ClearCase and even Team Foundation Server. Saturday, July 24, 2010
  19. 19. Notifications • Hudson has built-in notifications via email • Via user integration, it can also email users who have broken or fixed the build • Many plugins exist for other notification media, such as multi-protocol IM and IRC, suitable for running in a team IRC room • If you enter committers IRC nicks / IM handles, the bot will praise or chastise them Saturday, July 24, 2010
  20. 20. Slave Builds • Hudson can install a Java-based slave via SSH on any *nix machine • The slave will run builds on the local machine and report build status back to Hudson • Jobs can be tied to any open slave (better for jobs like running cron) or to a specific machine (often used for deployments) Saturday, July 24, 2010
  21. 21. Build Steps Credit: Saturday, July 24, 2010
  22. 22. Build Steps • The meat of any job is the build steps that it goes through • When used with *nix machines, these build steps are shell scripts • Hudson sets a number of environmental variables when these jobs are run Saturday, July 24, 2010
  23. 23. Build Steps, cont’d • $WORKSPACE - the directory with the VCS code • $BUILD_URL - the URL to this specific build • $BUILD_NUMBER - the integer build number • In addition, you can create parameterized builds where a default value is set, but users running builds in the Hudson interface can specify a different value on a per-build basis Saturday, July 24, 2010
  24. 24. Build Steps, cont’d • When you look in the console for a build that runs, you’ll see that it runs sh -xe and then the scripts that you have put into the build steps • This option will make it so that if any step in the process returns a non-zero exit code, the build will fail Saturday, July 24, 2010
  25. 25. Testing • The most obvious use for testing is to run Drupal SimpleTests on every checkin • To do this, set up a job connected to your source control and that checks out your site • Install the site on the machine and make sure that the SimpleTest module is enabled Saturday, July 24, 2010
  26. 26. Testing, cont’d • With the SimpleTest module enabled, you can use the scripts/ script to run SimpleTests from the command line • These tests will be run as the Hudson user when run through Hudson, so that’s where the umask 002 we set up earlier is important Saturday, July 24, 2010
  27. 27. Testing, cont’d • The SimpleTest module in Pressflow 6 contains a patch to allow the SimpleTest script to output test results in the JUnit XML format using the --xml switch • Hudson will show your test successes and failure and to make a build unstable if tests fail • Saturday, July 24, 2010
  28. 28. Linking Jobs Credit: Saturday, July 24, 2010
  29. 29. Other Scheduling Options • In addition to having a build run whenever there are new changes on a branch in source control, you can schedule builds: • Using a cron-like syntax • To run when another completes successfully • By hand or with a GET to /job/[job name]/build Saturday, July 24, 2010
  30. 30. Other Scheduling, cont’d • When you make a chain of builds, one failure in a build in the chain will stop the whole thing • You can also select to make a single unstable build stop the chain Saturday, July 24, 2010
  31. 31. Deployment Credit: Saturday, July 24, 2010
  32. 32. Deployment • Hudson is a great tool for automating deployments • A common workflow when working with DVCS systems is to have an integration branch or repository that changes get pushed to • When new changes appear in the integration repo, Hudson can check it and run tests Saturday, July 24, 2010
  33. 33. Deployment, cont’d • Using drush (and optionally its master/slave support) it is quite easy to automate updating other environments • Provided that all important schema updates are in code, drush can run all these automatically in a deploy task Saturday, July 24, 2010
  34. 34. Deployment, cont’d • The following script can be used as a deploy script on a local or remote machine: • drush -y updb • drush -y features-update-all • drush -y cc all Saturday, July 24, 2010
  35. 35. Other Cool Tricks Credit: Saturday, July 24, 2010
  36. 36. Run cron • David Strauss of Four Kitchens similarly elucidates on why Drush and Hudson rock and how you can use them together • cron-use-hudson-instead Saturday, July 24, 2010
  37. 37. View Status in NetBeans • NetBeans includes a Hudson plugin in core. Saturday, July 24, 2010
  38. 38. Track Back-End Performance • Make a build that does some GET requests against a production site • Set “XDEBUG_PROFILE” in $_GET, $_POST or $_COOKIE to get a cachegrind file • Run the cachegrind file through xdebugtoolkit to get a nice PDF to show where your execution time goes and track it over time. Saturday, July 24, 2010
  39. 39. Yellow items are called a lot, red items take a lot of time per call. Magenta items are both. Saturday, July 24, 2010
  40. 40. Track Front-End Performance • Install ShowSlow from • Use xtightvnc to make a virtual display that ShowSlow’s integration scripts can direct Firefox at on a set schedule • Compare your Google Page Speed and YSlow! scores over time Sorry, a magician doesn’t reveal his secrets, and I haven’t had enough time to write up these instructions yet. But it works - Trust Me™. Saturday, July 24, 2010
  41. 41. Thanks • The whole @hudsonci team for their work in creating this amazing tool. • Steven Jones for his work on JUnit XML output for SimpleTests. • The Pressflow team for including the patch in Pressflow 6. Community plumbing, baby. Saturday, July 24, 2010
  42. 42. Questions? Saturday, July 24, 2010