Your SlideShare is downloading. ×
0
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Learn About Continuous Integration With Hudson Directly From the Source

8,113

Published on

San Francisco Java User Group presents Kohsuke Kawaguchi from Sun who introduces us to Hudson, an open-source continuous integration (CI) system, which improves the productivity of a development team …

San Francisco Java User Group presents Kohsuke Kawaguchi from Sun who introduces us to Hudson, an open-source continuous integration (CI) system, which improves the productivity of a development team by automating various things.

Hosted By: SUPINFO International University
Sponsored By: TEKsystems, Sun, O'Reilly, JetBrains, and Marakana
Organized By: Marakana
Video By: Max Walker

Video available at http://marakana.com/forums/java/general/79.html

Published in: Education
1 Comment
13 Likes
Statistics
Notes
No Downloads
Views
Total Views
8,113
On Slideshare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
897
Comments
1
Likes
13
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Plugins --- up 80 from last year in JavaOne, contributers – up 70 from last year
  • Demo cheat sheet:Ant project build. Build unit test. Polling.Maven project. Auto configuration. Javadoc.Plugin manager, update center
  • We’ve seen that how Hudson implements different launch protocol for Windows and slaves, but that’s not the only place where we take heterogeneous environments seriously.
  • Transcript

    1. HudsonSF JUG<br />Kohsuke Kawaguchi<br />Sun Microsystems, Inc.<br />
    2. Rise of Continuous Integration<br />Offload from people, push to computers<br />$<br />computers<br />people<br />time<br />2<br />
    3. Spend more CPU power to help you<br />First on your laptops and workstations<br />IDEs are at the forefront<br />And then to the servers<br />a.k.a. “Continuous Integration”<br />More frequent build/test executions<br />Static code analysis tools<br />And more to come<br />3<br />
    4. Hudson<br />Open-source CI server<br />Emphasis on ease of installation and use<br />“java -jar hudson.war” execution<br />Or your OS-specific package<br />Configure everything from browsers<br />Extensibility<br />230+ community-developed public plugins<br />By 230+ contributors<br />Estimated 13,000 installations<br />4<br />https://hudson-ci.org/<br />
    5. Adoption in all kinds of businesses<br />5<br />
    6. Support Subscriptions<br />Community releases every week<br />Over 340 releases to date<br />Rapid new features & bug fixes delivery<br />“Sun Continuous Integration Server”<br />For enterprise that needs stability<br />Bug fix only patch releases for 18 months<br />6<br />
    7. Basic Feature Set<br />Hudson notices that there is a change<br />Check out the source code<br />CVS, Subversion, ClearCase, Mercurial, Accurev, Perforce, Git, StarTeam, …<br />Do builds<br />Ant, Maven, shell script, NAnt, …<br />Record and publish results<br />Close the feedback loop<br />RSS feeds<br />E-mail/IM notification<br />Tray application to notify developers<br />Tool integration<br />Findbugs, cobertura, …<br />7<br />
    8. Demo<br />
    9. Why do I care?<br />Life before Hudson<br />Dev forgets to commit a file<br />RE nightly build breaks. It blocks SQE<br />People waste time<br />Life after Hudson<br />Hudson will tell you in 5 mins if a build broke<br />No one else notices that you forgot to commit a file<br />9<br />
    10. Why do I care?<br />Life before Hudson<br />Dev makes a change Monday AM<br />SQE nightly finds a bug Monday night<br />Dev fixes it Tuesday<br />Lot of wasted time<br />Life after Hudson<br />Dev makes a change Monday AM<br />Hudson finds a regression 30 mins later<br />Dev can fix it before lunch<br />10<br />
    11. Why do I care?<br />Life before Hudson<br />SQE test runs every night, results sent out in e-mail<br />After the enthusiasm of the 1st week is gone, nobody looks at them anymore<br />Regressions go unnoticed until it’s too late<br />Life after Hudson<br />Tests run Hudson after every commit<br />E-mail sent out only when tests start failing<br />So it manages to keep people’s attention<br />11<br />
    12. Matrix projects<br />Run the same thing on different environments<br />such as different JDKs, different databases, and different OSes<br />Aggregate the results<br />12<br />
    13. FindBugs integration<br />13<br />
    14. Tracking changes<br />14<br />
    15. … and more<br />Browse workspaces<br />Build time trend report<br />15<br />
    16. Inter-Project Support<br />You can host many projects on 1 Hudson<br />And when you do, 1+1 becomes more than 2<br />You can…<br />Execute tests separately and correlate them with builds<br />Mark builds as “promoted” based on test outcome<br />Track which build of libraries are used where<br />And more<br />16<br />
    17. Test/Build Separation<br />Tests often dominate the build time<br />Multiple tests are run in parallel<br />Improves turn-around time<br />time<br />JAX-WS #10<br />JAX-WS #11<br />JAX-WS #12<br />…<br />Unit test with GF<br />Unit test with Tomcat<br />SQE test<br />Test w/o container<br />3OS x 2JDK = 6 tests<br />17<br />
    18. Test/Build Separation Technique<br />Copy bits from the build to test via wget by permalinks<br />Ant has the equivalent &lt;get&gt; task<br />“Fingerprinting” to keep track of where binaries are used<br />$ wget –O foo.zip<br /> http://server/hudson/job/foo/lastSuccessfulBuild/ artifact/foo.zip<br />18<br />
    19. Build Promotion<br />CI produces a lot of “successful” builds<br />Often overwhelming to downstream consumers<br />Run tests as fast as you can<br />If a build passes tests, promote it<br />19<br />
    20. Got the idea?<br />Automation<br />Reduce turn-around time<br />Make things transparent<br />Remove people from the loop<br />Save people’s time<br />Push jobs to servers, keep workstations idle for you<br />20<br />
    21. Doing Distributed Builds<br />21<br />
    22. Why?<br />There’s only so much 1 computer can do<br />But lots of PCs = tedious<br />What does Hudson do to help you?<br />22<br />
    23. Installing new slaves<br />For first 20 or so slaves, we did it manually<br />Insert CD, click, type, click, type, click, …<br />But that doesn’t scale<br />Then we automated<br />Available as “Hudson PXE Plugin”<br />23<br />
    24. Automated System Installations<br /><ul><li>Hudson + PXEplugin
    25. ISO images of OS</li></ul>Slaves<br />Power on, hit F12<br />PC boots from network (PXE)<br />24<br />
    26. Automated System Installations<br /><ul><li>Hudson + PXEplugin
    27. ISO images of OS</li></ul>Your corporate IT guy & his DHCP server<br />Slaves<br />Power on, hit F12<br />PC boots from network (PXE)<br />Choose OS from menu<br />Installs non-interactively<br />25<br />
    28. Distributed builds with Hudson<br />Master<br />Serves HTTP requests<br />Stores all important info<br />Slaves<br />170KB single JAR<br />Assumed to be unreliable<br />Scale to at least 100<br />Link<br />Single bi-di byte stream<br />No other requirements<br />26<br />
    29. How master and slaves start talking<br />For Unix slaves, via SSH<br />Only need SSHD and JRE on slaves<br />We just need a host name<br />27<br />
    30. How master and slaves start talking<br />For Windows, DCOM<br />We just need admin user name and password<br />No manual intervention<br />Works even from Unix masters<br />28<br />
    31. How master and slaves start talking<br />Via Java Web Start<br />When master cannot see slaves<br />A separate socket connection is made<br />29<br />
    32. Automating JNLP launch<br />Once started, can be installed as Windows service<br />30<br />
    33. Automating JNLP launch<br />Emulate the JNLP client headless<br />$ java -jar slave.jar -jnlpUrl URL<br />31<br />
    34. Automated Tool Installation - JDK<br />JDK from http://java.sun.com/<br />Hudson automatically chooses the right bundle<br />Always up to date with new releases<br />32<br />
    35. Automated Tool Installation - Apache<br />Ant and Maven from Apache<br />33<br />
    36. Heterogeneous Cluster Challenge<br />Your builds/tests need to run in specific environment<br />Dependency on individual nodes hurts utilization<br />jobs<br />slaves<br />WombatWindows test<br />Windows #1<br />GlassFishWindows test<br />Windows #2<br />Hudson Windows test<br />Solaris#1<br />Hudson Solaris test<br />34<br />
    37. Labels to rescue<br />Label is a group of slaves<br />Tie jobs to labels<br />jobs<br />slaves<br />WombatWindows test<br />Windows #1<br />Windows<br />GlassFishWindows test<br />Windows #2<br />Hudson Windows test<br />Solaris#1<br />Solaris<br />Hudson Solaris test<br />Windows #3<br />35<br />
    38. Concurrent Builds<br />36<br />Hudson #10<br />Hudson #11<br />Hudson #12<br />Hudson #13<br />Hudson #14<br />Hudson #15<br />time<br />
    39. Making builds sticky<br />We want jobs to be mostly on the same slave<br />Faster check out<br />Consistent results<br />Minimizes disk consumption<br />But does it softly<br />Hudson uses consistent hash* to achieve this<br />More schedule controls become possible:<br />Use faster machines more frequently<br />Slowly ramp up newly installed slaves<br />* http://en.wikipedia.org/wiki/Consistent_hashing<br />37<br />
    40. Forecasting failures<br />Hudson monitors key health metrics of slaves<br />Low disk space, insufficient swap<br />Clock out of sync<br />Extensible<br />Slaves go offline automatically<br />38<br />
    41. Clean up mess after builds<br />Kill run-away processes<br />Daemons, background processes left by your build<br />Works on Windows, Linux, Mac, and Solaris<br />39<br />
    42. Load Statistics Monitoring<br />40<br />
    43. When it’s time to add more slaves<br />41<br />
    44. Hudson EC2 plugin<br />Automatically provisions slaves on EC2 on demand<br />Based on the current load<br />Picks the right AMI depending on demand<br />Starts slave agent<br />Shuts down unused instances<br />42<br />
    45. Putting it all together<br />capacity<br /># of executors<br />usage<br />queue length<br />time<br />43<br />
    46. Hudson “Appliance” on EC2<br />Run the master in the cloud too, if you like<br />Hudson on stock OpenSolaris AMI<br />Data stored persistently in Elastic Block Storage<br />Dynamically expandable thanks to ZFS<br />Online, too<br />Packaged as a wizard<br />44<br />
    47. And a lot more I couldn’t show<br />IDE plugins<br />iPhone/Android apps<br />REST API and CLI<br />Deployment Automation<br />Hadoop, Selenium<br />…<br />45<br />
    48. Conclusions<br />CI is here to stay<br />Using lots of PCs is a key<br />Hudson is very easy to get started<br />Once started, there are a lot you can do<br />46<br />
    49. Q&A<br />Kohsuke Kawaguchi<br />Sun Microsystems, Inc.<br />
    50. Compartmentalization<br />full Metro build #10<br />full Metro build #11<br />JAXB #10<br />JAXB #11<br />JAXB #12<br />Faster feedback<br />Good engineering practice anyway<br />Isolation of problems<br />JAX-WS #20<br />JAX-WS #21<br />JAX-WS #22<br />time<br />48<br />
    51. Automated System Installations<br />Supports OpenSolaris, Ubuntu, CentOS, Fedora<br />Trivial with most Linux<br />Cooperate with Windows, too<br />Quite useful outside Hudson, too<br />No more broken CD drives<br />No more CD-Rs<br />49<br />

    ×