Hudson at FISL 2009

2,192 views
2,131 views

Published on

Continuous Integration using Hudson

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,192
On SlideShare
0
From Embeds
0
Number of Embeds
97
Actions
Shares
0
Downloads
86
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Hudson at FISL 2009

  1. 1. Continuous Integration with Hudson Arun Gupta Sun Microsystems, Inc.
  2. 2. Acknowledgements Originally presented by Kohsuke Kawaguchi & Jesse Glick at JavaOne Scheduled to be presented by Fabiane Nardon At FISL 10
  3. 3. Rise of Continuous Integration > Offload from people, push to computers $ computers us time 3
  4. 4. “Never use humans for the job of a machine”
  5. 5. Spend more CPU power to help you > … even if it only helps a little > First on your laptops and workstations  IDEs are at the forefront > And then to the servers  a.k.a. “Continuous Integration”  More frequent build/test executions  Static code analysis tools  And more to come 5
  6. 6. Hudson https://hudson.dev.java. net/ > Open-source CI server at java.net > Emphasis on ease of installation and use  “java -jar hudson.war” execution  Configure everything from browsers > Extensibility  140+ community-developed public plugins  By 150+ contributors > Estimated 13,000 installations 6
  7. 7. It basically does builds and tests > Check out the source code  Subversion, Perforce, Git, Mercurial, CVS, … > Do builds and/or tests  Ant, Maven, MSBuild, shell script, … > Record results  Binary, test results, code coverage, static analysis > Notify people  E-mail, IM, RSS, tray apps, IDEs 7
  8. 8. Hudson Plugin Ecosystem New Plugins 2008: 55 2009: 44 so far 8
  9. 9. Localized to 8 languages 9
  10. 10. And hopefully more to come… 10
  11. 11. Adoption in all kinds of businesses 11
  12. 12. Eclipse Community Survey 12
  13. 13. 13
  14. 14. Why Distributed Builds? > You need to use multiple computers because…  You need different environments  You need isolation > There’s only so much you can do with 1 computer 14
  15. 15. Installing new slaves > For first 20 or so slaves, we did it manually  Insert CD, click, type, click, type, click, …  But that doesn’t scale > Then we automated  Available as “Hudson PXE Plugin” 15
  16. 16. Automated System Installations > Hudson + PXE plugin  ISO images of OS > Slaves  Power on, hit F12  PC boots from network (PXE) 16
  17. 17. Automated System Installations > Hudson + PXE plugin  ISO images of OS Your corporate IT guy & his DHCP server > Slaves  Power on, hit F12  PC boots from network (PXE)  Choose OS from menu  Installs non-interactively 17
  18. 18. Automated System Installations > Supports OpenSolaris, Ubuntu, CentOS, Fedora  Trivial with most Linux > Cooperate with Windows, too > Quite useful outside Hudson, too  No more broken CD drives  No more CD-Rs 18
  19. 19. Distributed builds with Hudson > Master  Serves HTTP requests  Stores all important info > Slaves  170KB single JAR  Assumed to be unreliable  Scale to at least 100 > Link  Single bi-di byte stream  No other requirements 19
  20. 20. Heterogeneous Cluster Challenge > Your builds/tests need to run in specific environment > Dependency on individual nodes hurts utilization jobs slaves Wombat Window Windows test s #1 GlassFish Window Windows test s #2 Hudson Solaris Windows test #1 Hudson Solaris test 20
  21. 21. Labels to rescue > Label is a group of slaves > Tie jobs to labels jobs slaves Wombat Window Windows test s #1 Windows GlassFish Window Windows test s #2 Hudson Solaris Windows test Solaris #1 Hudson Solaris Window test s #3 21
  22. 22. Coming to your Hudson Making builds sticky soon > We want jobs to be mostly on the same slave  Faster check out  Consistent results  Minimizes disk consumption > But does it softly > Hudson uses consistent hash* to achieve this > More schedule controls become possible:  Use faster machines more frequently  Slowly ramp up newly installed slaves * http://en.wikipedia.org/wiki/Consistent_hashing 22
  23. 23. Forecasting failures > Hudson monitors key health metrics of slaves  Low disk space, insufficient swap  Clock out of synch  Extensible > Slaves go offline automatically > Catch problems before they break builds 23
  24. 24. Load Statistics Monitoring 24
  25. 25. When it’s time to add more slaves 25
  26. 26. Hudson made this extensible > Hudson detects excessive workload > Hudson notifies plugins > Plugins can provision more slaves  … assuming that you have that infrastructure 26
  27. 27. 27
  28. 28. Amazon EC2: The Good > Pay as you go (10¢/h or so)  Loads on Hudson tend to be spiky > Programmable API > Instances launch at machine-speed > EC2 instances are forgetful 28
  29. 29. Amazon EC2: The Bad > Your data is still inside your firewall  Takes time to check out code  … or to archive build artifacts  Some data just can’t be moved > EC2 instances are forgetful > Can your tests run in parallel? 29
  30. 30. Hudson EC2 plugin > Built on top of typica* > What does it do?  Automatically provisions slaves on EC2 on demand  Picks the right AMI depending on demand  Starts slave agent  Shuts down unused instances * http://code.google.com/p/typica/ 30
  31. 31. Hudson “Appliance” on EC2 > Run the master in the cloud too, if you like  Hudson on stock OpenSolaris AMI  Data stored persistently in Elastic Block Storage  Dynamically expandable thanks to ZFS  Online, too > Packaged as a wizard 31
  32. 32. Hudson EC2 plugin usage > Tell Hudson what AMIs you want to start 32
  33. 33. 33
  34. 34. Conclusion > CI is here to stay  We’ll continue to push more workload to servers > Hudson makes this easy for you > Reap the benefit of a cluster in multiple ways 34
  35. 35. Resources > http://hudson.dev.java.net/ > Support Subscription  hudson@sun.com 35
  36. 36. Arun Gupta http://blogs.sun.com/arungupta http://hudson.dev.java.net/ 36

×