This document discusses Continuous Integration with Hudson. It describes how Hudson automates building and testing software projects to help catch bugs early. Hudson can distribute builds across multiple computers called slaves to improve performance. Plugins allow Hudson to integrate with services like Amazon EC2 to dynamically provision more build resources as demand increases. The document promotes Hudson as an easy to use open source tool for Continuous Integration.
5. Spend more CPU power to help you
> … even if it only helps a little
> First on your laptops and workstations
IDEs are at the forefront
> And then to the servers
a.k.a. “Continuous Integration”
More frequent build/test executions
Static code analysis tools
And more to come
5
6. Hudson https://hudson.dev.java.
net/
> Open-source CI server at java.net
> Emphasis on ease of installation and use
“java -jar hudson.war” execution
Configure everything from browsers
> Extensibility
140+ community-developed public plugins
By 150+ contributors
> Estimated 13,000 installations
6
7. It basically does builds and tests
> Check out the source code
Subversion, Perforce, Git, Mercurial, CVS, …
> Do builds and/or tests
Ant, Maven, MSBuild, shell script, …
> Record results
Binary, test results, code coverage, static analysis
> Notify people
E-mail, IM, RSS, tray apps, IDEs
7
14. Why Distributed Builds?
> You need to use multiple computers because…
You need different environments
You need isolation
> There’s only so much you can do with 1 computer
14
15. Installing new slaves
> For first 20 or so slaves, we did it manually
Insert CD, click, type, click, type, click, …
But that doesn’t scale
> Then we automated
Available as “Hudson PXE Plugin”
15
16. Automated System Installations
> Hudson + PXE plugin
ISO images of OS
> Slaves
Power on, hit F12
PC boots from network (PXE)
16
17. Automated System Installations
> Hudson + PXE plugin
ISO images of OS
Your corporate IT guy &
his DHCP server
> Slaves
Power on, hit F12
PC boots from network (PXE)
Choose OS from menu
Installs non-interactively
17
18. Automated System Installations
> Supports OpenSolaris, Ubuntu, CentOS, Fedora
Trivial with most Linux
> Cooperate with Windows, too
> Quite useful outside Hudson, too
No more broken CD drives
No more CD-Rs
18
19. Distributed builds with Hudson
> Master
Serves HTTP requests
Stores all important info
> Slaves
170KB single JAR
Assumed to be unreliable
Scale to at least 100
> Link
Single bi-di byte stream
No other requirements
19
20. Heterogeneous Cluster Challenge
> Your builds/tests need to run in specific
environment
> Dependency on individual nodes hurts utilization
jobs slaves
Wombat Window
Windows test
s #1
GlassFish Window
Windows test
s #2
Hudson Solaris
Windows test
#1
Hudson Solaris
test
20
21. Labels to rescue
> Label is a group of slaves
> Tie jobs to labels
jobs slaves
Wombat Window
Windows test
s #1
Windows
GlassFish Window
Windows test
s #2
Hudson Solaris
Windows test
Solaris #1
Hudson Solaris Window
test
s #3
21
22. Coming to
your
Hudson
Making builds sticky soon
> We want jobs to be mostly on the same slave
Faster check out
Consistent results
Minimizes disk consumption
> But does it softly
> Hudson uses consistent hash* to achieve this
> More schedule controls become possible:
Use faster machines more frequently
Slowly ramp up newly installed slaves
* http://en.wikipedia.org/wiki/Consistent_hashing 22
23. Forecasting failures
> Hudson monitors key health metrics of slaves
Low disk space, insufficient swap
Clock out of synch
Extensible
> Slaves go offline automatically
> Catch problems before they break builds
23
26. Hudson made this extensible
> Hudson detects excessive workload
> Hudson notifies plugins
> Plugins can provision more slaves
… assuming that you have that infrastructure
26
28. Amazon EC2: The Good
> Pay as you go (10¢/h or so)
Loads on Hudson tend to be spiky
> Programmable API
> Instances launch at machine-speed
> EC2 instances are forgetful
28
29. Amazon EC2: The Bad
> Your data is still inside your firewall
Takes time to check out code
… or to archive build artifacts
Some data just can’t be moved
> EC2 instances are forgetful
> Can your tests run in parallel?
29
30. Hudson EC2 plugin
> Built on top of typica*
> What does it do?
Automatically provisions slaves on EC2 on demand
Picks the right AMI depending on demand
Starts slave agent
Shuts down unused instances
* http://code.google.com/p/typica/ 30
31. Hudson “Appliance” on EC2
> Run the master in the cloud too, if you like
Hudson on stock OpenSolaris AMI
Data stored persistently in Elastic Block Storage
Dynamically expandable thanks to ZFS
Online, too
> Packaged as a wizard
31
32. Hudson EC2 plugin usage
> Tell Hudson what AMIs you want to start
32
34. Conclusion
> CI is here to stay
We’ll continue to push more workload to servers
> Hudson makes this easy for you
> Reap the benefit of a cluster in multiple ways
34
35. Resources
> http://hudson.dev.java.net/
> Support Subscription
hudson@sun.com
35