Continuous Integration
with Hudson
Javaday Torino, 20/10/2007
Carlo Bonamico
carlo.bonamico@eptamedia.com
juggenova@gmail.com
http://www.juggenova.net
1

Overview
• Introduction to Continuous Integration
• CI workflow and practices
• Hudson
– applying practices with hudson
– DEMOS!
– architecture
• Tips and Tricks
2

Integration is hard!
• You already knew that! :-)
• Effort increase exponentially with
– number of bugs
– number of components
– time since last integration
• Ideally, we would like
to avoid it!
3

Why do we need CI?
• Replace big (and long) integration
phases with small and frequent ones
– ideally, instantaneous
– think of “continuous compilation” in Eclipse
• Minimize integration effort
– keep the development process running...
• CI was introduced with XP
recommended
– term introduced by M. Fowler in reading !
– http://www.martinfowler.com
/articles/continuousIntegration.html 4

Workflow
1.Checkout from SCM
2.Code a new feature
3.Run automated build on my machine
– repeat 2-3 until tests pass!
4.Merge with latest changes from SCM
– fix and rebuild until tests pass!
5.Commit
6.Run a build on a clean machine
– immediately fix bugs and integration issues!
5
20/10/2007 Genova JUG - http://www.genovajug.it

Benefits
• Finding bugs is easier
– self-testing builds
– diff debugging when you do not
know how long it
– less cumulative bugs will take to fix all
open bugs
• Reduced risks
– easier predictions
– avoid blind spots in the project
• Developers concentrate on coding
• Encourage more frequent deployments
6
20/10/2007 Genova JUG - http://www.genovajug.it

Fowler's 10 practices of CI
1)Maintain a single 1)Keep the build fast
source repository 2)Test in a clone of
2)Automate the build the production
environment
3)Make the build self-
testing 3)Make it easy to get
the last executable
4)Commit every day
4)Everyone can see
5)Every commit builds
what's happening
on an integration
machine 5)Automate
deployment
7
20/10/2007 Genova JUG - http://www.genovajug.it

Hudson build server
• Written in Java
– https://hudson.dev.java.net
• No install required!
– https://hudson.dev.java.net/hudson.jnlp
– java -jar hudson.war
– put war in web container
• Easy to use
thanks,
• Powerful
Kohsuke!
• Great support
8
20/10/2007 Genova JUG - http://www.genovajug.it

Single Source Repository
• A Version Control repository must
include ALL needed sources
– and nothing else...
– CVS, SVN, ...
• Issue: external dependencies
– approach “all in the repository”
• e.g. svn externals
– approach “dependency management” with
a second repository for binaries
• e.g. maven
9
20/10/2007 Genova JUG - http://www.genovajug.it

SCM in hudson
• Default support for CVS and SVN
– authentication
– svn:externals
10
20/10/2007 Genova JUG - http://www.genovajug.it

Automate the build
• What is a build?
• The entire set of ALL steps needed to
obtain the software product
– not just compiling…
11
20/10/2007 Genova JUG - http://www.genovajug.it

Build inputs
• Sources
– Java ;-)
– multilanguage (C/C++, scripts...)
• Configuration files
– properties, XML
• Resources
– images
– text, HTML help
–… 12
20/10/2007 Genova JUG - http://www.genovajug.it

Build output
• Exceutable / library
– jar / exe / dll
• Documentation
– javadoc, doxygen, xdoc
• Installers
– jar, msi, xpi ...
• Source Package
– zip
13
20/10/2007 Genova JUG - http://www.genovajug.it

How to automate it?
• Define the build in a configuration file
• Delegate the work to a build engine
• makefile
• ant
• maven
• rake
– custom script (e.g. shell, python)
• Issue: sync with the IDE
14
20/10/2007 Genova JUG - http://www.genovajug.it

Every commit triggers a build
• Commit at least daily...
– as soon as you have completed an
independent functionality
• A full build on another, “empty” machine
– not on your own!
• With hudson
– SCM polling
– cron-like scheduler
– trigger with HTTP request
• e.g. from ant or svn commit hook
15
20/10/2007 Genova JUG - http://www.genovajug.it

Self-testing build
• Directly go from source to running tests
– no manual copying
– no click on dialog boxes
– no configuration file editing
• Test with
– XUnit family
• jUnit, UnitTest++, ...
– Functional testing
• FIT, Selenium, Watir, ...
16
20/10/2007 Genova JUG - http://www.genovajug.it

Test reporting with hudson
• Automatic publishing of
– xUnit reports
– code coverage
– open tasks
• Builds can be
– successful
– unstable
– failed
17
20/10/2007 Genova JUG - http://www.genovajug.it

Test on a clone of the
production machine
• Hudson can be extended with plugins for
– deploying files to a remote machine via scp
– launching vmware instances
• This requires the setup of automated
delopyment
– easy in Java with war files...
18
20/10/2007 Genova JUG - http://www.genovajug.it

Everybody sees what's
happening
• Web based interface
• Notifications via
– email
– IM (jabber)
– RSS
– Blame mechanism
• Eclipse plugin
– http://code.google.com/p/hudson-eclipse/
– plugins for other IDEs
19
20/10/2007 Genova JUG - http://www.genovajug.it

Anyone can get the latest
executable
• Each project defines artifacts that are
published on the web interface
– exe
– jar
– war
– ...
20
20/10/2007 Genova JUG - http://www.genovajug.it

Keep the build fast
• Hudson provides two mechanisms to
make big builds more manageable
• Pipelined builds
– split build and functional testing in separate
jobs
– run them in cascade
• Clustered builds with remote slaves
– added quickly through JNLP or ssh
– unified dashboard on the master
21
20/10/2007 Genova JUG - http://www.genovajug.it

Advanced features
• Fingerprinting
– verify the sources and build number for an
artifact
• Build C/C++ projects
– Visual Studio command line interface
– hudson Build plugins for
• nant
• msbuild
• Run and monitor jobs
– schedule backups and mirroring
• e.g. rsync in a bash script
22
20/10/2007 Genova JUG - http://www.genovajug.it

Architecture
• Stapler library for meaningful URL
mappings
/hudson/job/SimpleProject/build
– becomes
Hudson.getJob(“SimpleProject”).build()
• Jelly for the view layer
• Configuration is stored on XML files
– can be changed with the server running
– “reload settings” action
23
20/10/2007 Genova JUG - http://www.genovajug.it

Configuration
• HUDSON_HOME • Structure
config.xml
– ${user.home}/.hudson
*.xml
• overridden by fingerprints
plugins
– environment jobs
variable [JOBNAME]
config.xml
– system property workspace
latest
– JNDI variable builds
[ID]
build.xml
log
changelog.xml
24
20/10/2007 Genova JUG - http://www.genovajug.it

Extending hudson
• Plugins can add to the following
extension points
– SCM
• e.g. perforce, clearcase, polarion, vss
– Job and Builder
• e.g. gant, msbuild
– Trigger
– BuildWrapper
– Publisher
• e.g. emma, findbugs, clover, tasks
– Action 25
20/10/2007 Genova JUG - http://www.genovajug.it

Security
• Default: open to all
– WARNING: source code can be accessed
through a browser...
– WARNING: build scripts (and committers)
have full access to the local systesm
• Hudson internal “security enabled” mode:
– read-write access for authenticated users
– read-only for everyone else
• Additional restrictions with container config
26
20/10/2007 Genova JUG - http://www.genovajug.it

Other CI servers
• http://cruisecontrol.sf.net/
– the first…
• http://maven.apache.org/continuum/
– focused on MAVEN
– powerful but more complex UI
• http://www.atlassian.com/software/bamboo/
– commercial product
• http://damagecontrol.codehaus.org/Continuous+Integr
ation+Server+Feature+Matrix
27
20/10/2007 Genova JUG - http://www.genovajug.it

Conclusions
• Hudson is an easy-to-use but powerful
continuous integration server offering
– interface to a range of SCM systems
– support for ant, maven, shell-based builds
– comprehensive project analysis
– plugins that integrate with external tools
• Effective tool for introducing CI in your
project
28
20/10/2007 Genova JUG - http://www.genovajug.it

Thanks for your attention!
• Hudson
– http://hudson.dev.java.net
– http://weblogs.java.net/blog/kohsuke
• JUG Genova
– http://www.juggenova.net
• My Blog
– http://www.carlobonamico.com/blog
• My company
– http://www.eptamedia.com
29
20/10/2007 Genova JUG - http://www.genovajug.it