Continuous Integration With Hudson

Slide 1: Continuous Integration with Hudson Javaday Torino, 20/10/2007 Carlo Bonamico carlo.bonamico@eptamedia.com juggenova@gmail.com http://www.juggenova.net 1

Slide 2: Overview • Introduction to Continuous Integration • CI workflow and practices • Hudson – applying practices with hudson – DEMOS! – architecture • Tips and Tricks 2

Slide 3: Integration is hard! • You already knew that! :-) • Effort increase exponentially with – number of bugs – number of components – time since last integration • Ideally, we would like to avoid it! 3

Slide 4: Why do we need CI? • Replace big (and long) integration phases with small and frequent ones – ideally, instantaneous – think of “continuous compilation” in Eclipse • Minimize integration effort – keep the development process running... • CI was introduced with XP recommended – term introduced by M. Fowler in reading ! – http://www.martinfowler.com /articles/continuousIntegration.html 4

Slide 5: Workflow 1.Checkout from SCM 2.Code a new feature 3.Run automated build on my machine – repeat 2-3 until tests pass! 4.Merge with latest changes from SCM – fix and rebuild until tests pass! 5.Commit 6.Run a build on a clean machine – immediately fix bugs and integration issues! 5 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 6: Benefits • Finding bugs is easier – self-testing builds – diff debugging when you do not know how long it – less cumulative bugs will take to fix all open bugs • Reduced risks – easier predictions – avoid blind spots in the project • Developers concentrate on coding • Encourage more frequent deployments 6 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 7: Fowler's 10 practices of CI 1)Maintain a single 1)Keep the build fast source repository 2)Test in a clone of 2)Automate the build the production environment 3)Make the build self- testing 3)Make it easy to get the last executable 4)Commit every day 4)Everyone can see 5)Every commit builds what's happening on an integration machine 5)Automate deployment 7 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 8: Hudson build server • Written in Java – https://hudson.dev.java.net • No install required! – https://hudson.dev.java.net/hudson.jnlp – java -jar hudson.war – put war in web container • Easy to use thanks, • Powerful Kohsuke! • Great support 8 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 9: Single Source Repository • A Version Control repository must include ALL needed sources – and nothing else... – CVS, SVN, ... • Issue: external dependencies – approach “all in the repository” • e.g. svn externals – approach “dependency management” with a second repository for binaries • e.g. maven 9 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 10: SCM in hudson • Default support for CVS and SVN – authentication – svn:externals 10 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 11: Automate the build • What is a build? • The entire set of ALL steps needed to obtain the software product – not just compiling… 11 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 12: Build inputs • Sources – Java ;-) – multilanguage (C/C++, scripts...) • Configuration files – properties, XML • Resources – images – text, HTML help –… 12 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 13: Build output • Exceutable / library – jar / exe / dll • Documentation – javadoc, doxygen, xdoc • Installers – jar, msi, xpi ... • Source Package – zip 13 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 14: How to automate it? • Define the build in a configuration file • Delegate the work to a build engine • makefile • ant • maven • rake – custom script (e.g. shell, python) • Issue: sync with the IDE 14 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 15: Every commit triggers a build • Commit at least daily... – as soon as you have completed an independent functionality • A full build on another, “empty” machine – not on your own! • With hudson – SCM polling – cron-like scheduler – trigger with HTTP request • e.g. from ant or svn commit hook 15 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 16: Self-testing build • Directly go from source to running tests – no manual copying – no click on dialog boxes – no configuration file editing • Test with – XUnit family • jUnit, UnitTest++, ... – Functional testing • FIT, Selenium, Watir, ... 16 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 17: Test reporting with hudson • Automatic publishing of – xUnit reports – code coverage – open tasks • Builds can be – successful – unstable – failed 17 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 18: Test on a clone of the production machine • Hudson can be extended with plugins for – deploying files to a remote machine via scp – launching vmware instances • This requires the setup of automated delopyment – easy in Java with war files... 18 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 19: Everybody sees what's happening • Web based interface • Notifications via – email – IM (jabber) – RSS – Blame mechanism • Eclipse plugin – http://code.google.com/p/hudson-eclipse/ – plugins for other IDEs 19 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 20: Anyone can get the latest executable • Each project defines artifacts that are published on the web interface – exe – jar – war – ... 20 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 21: Keep the build fast • Hudson provides two mechanisms to make big builds more manageable • Pipelined builds – split build and functional testing in separate jobs – run them in cascade • Clustered builds with remote slaves – added quickly through JNLP or ssh – unified dashboard on the master 21 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 22: Advanced features • Fingerprinting – verify the sources and build number for an artifact • Build C/C++ projects – Visual Studio command line interface – hudson Build plugins for • nant • msbuild • Run and monitor jobs – schedule backups and mirroring • e.g. rsync in a bash script 22 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 23: Architecture • Stapler library for meaningful URL mappings /hudson/job/SimpleProject/build – becomes Hudson.getJob(“SimpleProject”).build() • Jelly for the view layer • Configuration is stored on XML files – can be changed with the server running – “reload settings” action 23 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 24: Configuration • HUDSON_HOME • Structure config.xml – ${user.home}/.hudson *.xml • overridden by fingerprints plugins – environment jobs variable [JOBNAME] config.xml – system property workspace latest – JNDI variable builds [ID] build.xml log changelog.xml 24 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 25: Extending hudson • Plugins can add to the following extension points – SCM • e.g. perforce, clearcase, polarion, vss – Job and Builder • e.g. gant, msbuild – Trigger – BuildWrapper – Publisher • e.g. emma, findbugs, clover, tasks – Action 25 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 26: Security • Default: open to all – WARNING: source code can be accessed through a browser... – WARNING: build scripts (and committers) have full access to the local systesm • Hudson internal “security enabled” mode: – read-write access for authenticated users – read-only for everyone else • Additional restrictions with container config 26 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 27: Other CI servers • http://cruisecontrol.sf.net/ – the first… • http://maven.apache.org/continuum/ – focused on MAVEN – powerful but more complex UI • http://www.atlassian.com/software/bamboo/ – commercial product • http://damagecontrol.codehaus.org/Continuous+Integr ation+Server+Feature+Matrix 27 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 28: Conclusions • Hudson is an easy-to-use but powerful continuous integration server offering – interface to a range of SCM systems – support for ant, maven, shell-based builds – comprehensive project analysis – plugins that integrate with external tools • Effective tool for introducing CI in your project 28 20/10/2007 Genova JUG - http://www.genovajug.it

Slide 29: Thanks for your attention! • Hudson – http://hudson.dev.java.net – http://weblogs.java.net/blog/kohsuke • JUG Genova – http://www.juggenova.net • My Blog – http://www.carlobonamico.com/blog • My company – http://www.eptamedia.com 29 20/10/2007 Genova JUG - http://www.genovajug.it