Hudson at FISL 2009
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Hudson at FISL 2009

on

  • 3,500 views

Continuous Integration using Hudson

Continuous Integration using Hudson

Statistics

Views

Total Views
3,500
Views on SlideShare
3,405
Embed Views
95

Actions

Likes
1
Downloads
82
Comments
0

6 Embeds 95

http://blogs.sun.com 54
http://weblogs.java.net 20
http://blog.arungupta.me 10
https://blogs.oracle.com 7
http://blogs.oracle.com 3
http://planets.sun.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Hudson at FISL 2009 Presentation Transcript

  • 1. Continuous Integration with Hudson Arun Gupta Sun Microsystems, Inc.
  • 2. Acknowledgements Originally presented by Kohsuke Kawaguchi & Jesse Glick at JavaOne Scheduled to be presented by Fabiane Nardon At FISL 10
  • 3. Rise of Continuous Integration > Offload from people, push to computers $ computers us time 3
  • 4. “Never use humans for the job of a machine”
  • 5. Spend more CPU power to help you > … even if it only helps a little > First on your laptops and workstations  IDEs are at the forefront > And then to the servers  a.k.a. “Continuous Integration”  More frequent build/test executions  Static code analysis tools  And more to come 5
  • 6. Hudson https://hudson.dev.java. net/ > Open-source CI server at java.net > Emphasis on ease of installation and use  “java -jar hudson.war” execution  Configure everything from browsers > Extensibility  140+ community-developed public plugins  By 150+ contributors > Estimated 13,000 installations 6
  • 7. It basically does builds and tests > Check out the source code  Subversion, Perforce, Git, Mercurial, CVS, … > Do builds and/or tests  Ant, Maven, MSBuild, shell script, … > Record results  Binary, test results, code coverage, static analysis > Notify people  E-mail, IM, RSS, tray apps, IDEs 7
  • 8. Hudson Plugin Ecosystem New Plugins 2008: 55 2009: 44 so far 8
  • 9. Localized to 8 languages 9
  • 10. And hopefully more to come… 10
  • 11. Adoption in all kinds of businesses 11
  • 12. Eclipse Community Survey 12
  • 13. 13
  • 14. Why Distributed Builds? > You need to use multiple computers because…  You need different environments  You need isolation > There’s only so much you can do with 1 computer 14
  • 15. Installing new slaves > For first 20 or so slaves, we did it manually  Insert CD, click, type, click, type, click, …  But that doesn’t scale > Then we automated  Available as “Hudson PXE Plugin” 15
  • 16. Automated System Installations > Hudson + PXE plugin  ISO images of OS > Slaves  Power on, hit F12  PC boots from network (PXE) 16
  • 17. Automated System Installations > Hudson + PXE plugin  ISO images of OS Your corporate IT guy & his DHCP server > Slaves  Power on, hit F12  PC boots from network (PXE)  Choose OS from menu  Installs non-interactively 17
  • 18. Automated System Installations > Supports OpenSolaris, Ubuntu, CentOS, Fedora  Trivial with most Linux > Cooperate with Windows, too > Quite useful outside Hudson, too  No more broken CD drives  No more CD-Rs 18
  • 19. Distributed builds with Hudson > Master  Serves HTTP requests  Stores all important info > Slaves  170KB single JAR  Assumed to be unreliable  Scale to at least 100 > Link  Single bi-di byte stream  No other requirements 19
  • 20. Heterogeneous Cluster Challenge > Your builds/tests need to run in specific environment > Dependency on individual nodes hurts utilization jobs slaves Wombat Window Windows test s #1 GlassFish Window Windows test s #2 Hudson Solaris Windows test #1 Hudson Solaris test 20
  • 21. Labels to rescue > Label is a group of slaves > Tie jobs to labels jobs slaves Wombat Window Windows test s #1 Windows GlassFish Window Windows test s #2 Hudson Solaris Windows test Solaris #1 Hudson Solaris Window test s #3 21
  • 22. Coming to your Hudson Making builds sticky soon > We want jobs to be mostly on the same slave  Faster check out  Consistent results  Minimizes disk consumption > But does it softly > Hudson uses consistent hash* to achieve this > More schedule controls become possible:  Use faster machines more frequently  Slowly ramp up newly installed slaves * http://en.wikipedia.org/wiki/Consistent_hashing 22
  • 23. Forecasting failures > Hudson monitors key health metrics of slaves  Low disk space, insufficient swap  Clock out of synch  Extensible > Slaves go offline automatically > Catch problems before they break builds 23
  • 24. Load Statistics Monitoring 24
  • 25. When it’s time to add more slaves 25
  • 26. Hudson made this extensible > Hudson detects excessive workload > Hudson notifies plugins > Plugins can provision more slaves  … assuming that you have that infrastructure 26
  • 27. 27
  • 28. Amazon EC2: The Good > Pay as you go (10¢/h or so)  Loads on Hudson tend to be spiky > Programmable API > Instances launch at machine-speed > EC2 instances are forgetful 28
  • 29. Amazon EC2: The Bad > Your data is still inside your firewall  Takes time to check out code  … or to archive build artifacts  Some data just can’t be moved > EC2 instances are forgetful > Can your tests run in parallel? 29
  • 30. Hudson EC2 plugin > Built on top of typica* > What does it do?  Automatically provisions slaves on EC2 on demand  Picks the right AMI depending on demand  Starts slave agent  Shuts down unused instances * http://code.google.com/p/typica/ 30
  • 31. Hudson “Appliance” on EC2 > Run the master in the cloud too, if you like  Hudson on stock OpenSolaris AMI  Data stored persistently in Elastic Block Storage  Dynamically expandable thanks to ZFS  Online, too > Packaged as a wizard 31
  • 32. Hudson EC2 plugin usage > Tell Hudson what AMIs you want to start 32
  • 33. 33
  • 34. Conclusion > CI is here to stay  We’ll continue to push more workload to servers > Hudson makes this easy for you > Reap the benefit of a cluster in multiple ways 34
  • 35. Resources > http://hudson.dev.java.net/ > Support Subscription  hudson@sun.com 35
  • 36. Arun Gupta http://blogs.sun.com/arungupta http://hudson.dev.java.net/ 36