Hudson at FISL 2009

Continuous Integration with Hudson Arun Gupta Sun Microsystems, Inc.

Acknowledgements Originally presented by Kohsuke Kawaguchi & Jesse Glick at JavaOne Scheduled to be presented by Fabiane Nardon At FISL 10

Rise of Continuous Integration > Offload from people, push to computers $ computers us time 3

“Never use humans for the job of a machine”

Spend more CPU power to help you > … even if it only helps a little > First on your laptops and workstations  IDEs are at the forefront > And then to the servers  a.k.a. “Continuous Integration”  More frequent build/test executions  Static code analysis tools  And more to come 5

Hudson https://hudson.dev.java. net/ > Open-source CI server at java.net > Emphasis on ease of installation and use  “java -jar hudson.war” execution  Configure everything from browsers > Extensibility  140+ community-developed public plugins  By 150+ contributors > Estimated 13,000 installations 6

It basically does builds and tests > Check out the source code  Subversion, Perforce, Git, Mercurial, CVS, … > Do builds and/or tests  Ant, Maven, MSBuild, shell script, … > Record results  Binary, test results, code coverage, static analysis > Notify people  E-mail, IM, RSS, tray apps, IDEs 7

Hudson Plugin Ecosystem New Plugins 2008: 55 2009: 44 so far 8

Localized to 8 languages 9

And hopefully more to come… 10

Adoption in all kinds of businesses 11

Eclipse Community Survey 12

Why Distributed Builds? > You need to use multiple computers because…  You need different environments  You need isolation > There’s only so much you can do with 1 computer 14

Installing new slaves > For first 20 or so slaves, we did it manually  Insert CD, click, type, click, type, click, …  But that doesn’t scale > Then we automated  Available as “Hudson PXE Plugin” 15

Automated System Installations > Hudson + PXE plugin  ISO images of OS > Slaves  Power on, hit F12  PC boots from network (PXE) 16

Automated System Installations > Hudson + PXE plugin  ISO images of OS Your corporate IT guy & his DHCP server > Slaves  Power on, hit F12  PC boots from network (PXE)  Choose OS from menu  Installs non-interactively 17

Automated System Installations > Supports OpenSolaris, Ubuntu, CentOS, Fedora  Trivial with most Linux > Cooperate with Windows, too > Quite useful outside Hudson, too  No more broken CD drives  No more CD-Rs 18

Distributed builds with Hudson > Master  Serves HTTP requests  Stores all important info > Slaves  170KB single JAR  Assumed to be unreliable  Scale to at least 100 > Link  Single bi-di byte stream  No other requirements 19

Heterogeneous Cluster Challenge > Your builds/tests need to run in specific environment > Dependency on individual nodes hurts utilization jobs slaves Wombat Window Windows test s #1 GlassFish Window Windows test s #2 Hudson Solaris Windows test #1 Hudson Solaris test 20

Labels to rescue > Label is a group of slaves > Tie jobs to labels jobs slaves Wombat Window Windows test s #1 Windows GlassFish Window Windows test s #2 Hudson Solaris Windows test Solaris #1 Hudson Solaris Window test s #3 21

Coming to your Hudson Making builds sticky soon > We want jobs to be mostly on the same slave  Faster check out  Consistent results  Minimizes disk consumption > But does it softly > Hudson uses consistent hash* to achieve this > More schedule controls become possible:  Use faster machines more frequently  Slowly ramp up newly installed slaves * http://en.wikipedia.org/wiki/Consistent_hashing 22

Forecasting failures > Hudson monitors key health metrics of slaves  Low disk space, insufficient swap  Clock out of synch  Extensible > Slaves go offline automatically > Catch problems before they break builds 23

Load Statistics Monitoring 24

When it’s time to add more slaves 25

Hudson made this extensible > Hudson detects excessive workload > Hudson notifies plugins > Plugins can provision more slaves  … assuming that you have that infrastructure 26

Amazon EC2: The Good > Pay as you go (10¢/h or so)  Loads on Hudson tend to be spiky > Programmable API > Instances launch at machine-speed > EC2 instances are forgetful 28

Amazon EC2: The Bad > Your data is still inside your firewall  Takes time to check out code  … or to archive build artifacts  Some data just can’t be moved > EC2 instances are forgetful > Can your tests run in parallel? 29

Hudson EC2 plugin > Built on top of typica* > What does it do?  Automatically provisions slaves on EC2 on demand  Picks the right AMI depending on demand  Starts slave agent  Shuts down unused instances * http://code.google.com/p/typica/ 30

Hudson “Appliance” on EC2 > Run the master in the cloud too, if you like  Hudson on stock OpenSolaris AMI  Data stored persistently in Elastic Block Storage  Dynamically expandable thanks to ZFS  Online, too > Packaged as a wizard 31

Hudson EC2 plugin usage > Tell Hudson what AMIs you want to start 32

Conclusion > CI is here to stay  We’ll continue to push more workload to servers > Hudson makes this easy for you > Reap the benefit of a cluster in multiple ways 34

Resources > http://hudson.dev.java.net/ > Support Subscription  hudson@sun.com 35

Arun Gupta http://blogs.sun.com/arungupta http://hudson.dev.java.net/ 36

http://blogs.sun.com	54
http://weblogs.java.net	20
http://blog.arungupta.me	9
https://blogs.oracle.com	7
http://blogs.oracle.com	3
http://planets.sun.com	1

Hudson at FISL 2009

by Arun Gupta

Accessibility

Categories

Upload Details

Usage Rights

6 Embeds 94

Statistics