Your SlideShare is downloading. ×
Learn About Continuous Integration With Hudson Directly From the Source
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Learn About Continuous Integration With Hudson Directly From the Source

8,070
views

Published on

San Francisco Java User Group presents Kohsuke Kawaguchi from Sun who introduces us to Hudson, an open-source continuous integration (CI) system, which improves the productivity of a development team …

San Francisco Java User Group presents Kohsuke Kawaguchi from Sun who introduces us to Hudson, an open-source continuous integration (CI) system, which improves the productivity of a development team by automating various things.

Hosted By: SUPINFO International University
Sponsored By: TEKsystems, Sun, O'Reilly, JetBrains, and Marakana
Organized By: Marakana
Video By: Max Walker

Video available at http://marakana.com/forums/java/general/79.html

Published in: Education

1 Comment
13 Likes
Statistics
Notes
No Downloads
Views
Total Views
8,070
On Slideshare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
895
Comments
1
Likes
13
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Plugins --- up 80 from last year in JavaOne, contributers – up 70 from last year
  • Demo cheat sheet:Ant project build. Build unit test. Polling.Maven project. Auto configuration. Javadoc.Plugin manager, update center
  • We’ve seen that how Hudson implements different launch protocol for Windows and slaves, but that’s not the only place where we take heterogeneous environments seriously.
  • Transcript

    • 1. HudsonSF JUG
      Kohsuke Kawaguchi
      Sun Microsystems, Inc.
    • 2. Rise of Continuous Integration
      Offload from people, push to computers
      $
      computers
      people
      time
      2
    • 3. Spend more CPU power to help you
      First on your laptops and workstations
      IDEs are at the forefront
      And then to the servers
      a.k.a. “Continuous Integration”
      More frequent build/test executions
      Static code analysis tools
      And more to come
      3
    • 4. Hudson
      Open-source CI server
      Emphasis on ease of installation and use
      “java -jar hudson.war” execution
      Or your OS-specific package
      Configure everything from browsers
      Extensibility
      230+ community-developed public plugins
      By 230+ contributors
      Estimated 13,000 installations
      4
      https://hudson-ci.org/
    • 5. Adoption in all kinds of businesses
      5
    • 6. Support Subscriptions
      Community releases every week
      Over 340 releases to date
      Rapid new features & bug fixes delivery
      “Sun Continuous Integration Server”
      For enterprise that needs stability
      Bug fix only patch releases for 18 months
      6
    • 7. Basic Feature Set
      Hudson notices that there is a change
      Check out the source code
      CVS, Subversion, ClearCase, Mercurial, Accurev, Perforce, Git, StarTeam, …
      Do builds
      Ant, Maven, shell script, NAnt, …
      Record and publish results
      Close the feedback loop
      RSS feeds
      E-mail/IM notification
      Tray application to notify developers
      Tool integration
      Findbugs, cobertura, …
      7
    • 8. Demo
    • 9. Why do I care?
      Life before Hudson
      Dev forgets to commit a file
      RE nightly build breaks. It blocks SQE
      People waste time
      Life after Hudson
      Hudson will tell you in 5 mins if a build broke
      No one else notices that you forgot to commit a file
      9
    • 10. Why do I care?
      Life before Hudson
      Dev makes a change Monday AM
      SQE nightly finds a bug Monday night
      Dev fixes it Tuesday
      Lot of wasted time
      Life after Hudson
      Dev makes a change Monday AM
      Hudson finds a regression 30 mins later
      Dev can fix it before lunch
      10
    • 11. Why do I care?
      Life before Hudson
      SQE test runs every night, results sent out in e-mail
      After the enthusiasm of the 1st week is gone, nobody looks at them anymore
      Regressions go unnoticed until it’s too late
      Life after Hudson
      Tests run Hudson after every commit
      E-mail sent out only when tests start failing
      So it manages to keep people’s attention
      11
    • 12. Matrix projects
      Run the same thing on different environments
      such as different JDKs, different databases, and different OSes
      Aggregate the results
      12
    • 13. FindBugs integration
      13
    • 14. Tracking changes
      14
    • 15. … and more
      Browse workspaces
      Build time trend report
      15
    • 16. Inter-Project Support
      You can host many projects on 1 Hudson
      And when you do, 1+1 becomes more than 2
      You can…
      Execute tests separately and correlate them with builds
      Mark builds as “promoted” based on test outcome
      Track which build of libraries are used where
      And more
      16
    • 17. Test/Build Separation
      Tests often dominate the build time
      Multiple tests are run in parallel
      Improves turn-around time
      time
      JAX-WS #10
      JAX-WS #11
      JAX-WS #12

      Unit test with GF
      Unit test with Tomcat
      SQE test
      Test w/o container
      3OS x 2JDK = 6 tests
      17
    • 18. Test/Build Separation Technique
      Copy bits from the build to test via wget by permalinks
      Ant has the equivalent <get> task
      “Fingerprinting” to keep track of where binaries are used
      $ wget –O foo.zip
      http://server/hudson/job/foo/lastSuccessfulBuild/ artifact/foo.zip
      18
    • 19. Build Promotion
      CI produces a lot of “successful” builds
      Often overwhelming to downstream consumers
      Run tests as fast as you can
      If a build passes tests, promote it
      19
    • 20. Got the idea?
      Automation
      Reduce turn-around time
      Make things transparent
      Remove people from the loop
      Save people’s time
      Push jobs to servers, keep workstations idle for you
      20
    • 21. Doing Distributed Builds
      21
    • 22. Why?
      There’s only so much 1 computer can do
      But lots of PCs = tedious
      What does Hudson do to help you?
      22
    • 23. Installing new slaves
      For first 20 or so slaves, we did it manually
      Insert CD, click, type, click, type, click, …
      But that doesn’t scale
      Then we automated
      Available as “Hudson PXE Plugin”
      23
    • 24. Automated System Installations
      • Hudson + PXEplugin
      • 25. ISO images of OS
      Slaves
      Power on, hit F12
      PC boots from network (PXE)
      24
    • 26. Automated System Installations
      • Hudson + PXEplugin
      • 27. ISO images of OS
      Your corporate IT guy & his DHCP server
      Slaves
      Power on, hit F12
      PC boots from network (PXE)
      Choose OS from menu
      Installs non-interactively
      25
    • 28. Distributed builds with Hudson
      Master
      Serves HTTP requests
      Stores all important info
      Slaves
      170KB single JAR
      Assumed to be unreliable
      Scale to at least 100
      Link
      Single bi-di byte stream
      No other requirements
      26
    • 29. How master and slaves start talking
      For Unix slaves, via SSH
      Only need SSHD and JRE on slaves
      We just need a host name
      27
    • 30. How master and slaves start talking
      For Windows, DCOM
      We just need admin user name and password
      No manual intervention
      Works even from Unix masters
      28
    • 31. How master and slaves start talking
      Via Java Web Start
      When master cannot see slaves
      A separate socket connection is made
      29
    • 32. Automating JNLP launch
      Once started, can be installed as Windows service
      30
    • 33. Automating JNLP launch
      Emulate the JNLP client headless
      $ java -jar slave.jar -jnlpUrl URL
      31
    • 34. Automated Tool Installation - JDK
      JDK from http://java.sun.com/
      Hudson automatically chooses the right bundle
      Always up to date with new releases
      32
    • 35. Automated Tool Installation - Apache
      Ant and Maven from Apache
      33
    • 36. Heterogeneous Cluster Challenge
      Your builds/tests need to run in specific environment
      Dependency on individual nodes hurts utilization
      jobs
      slaves
      WombatWindows test
      Windows #1
      GlassFishWindows test
      Windows #2
      Hudson Windows test
      Solaris#1
      Hudson Solaris test
      34
    • 37. Labels to rescue
      Label is a group of slaves
      Tie jobs to labels
      jobs
      slaves
      WombatWindows test
      Windows #1
      Windows
      GlassFishWindows test
      Windows #2
      Hudson Windows test
      Solaris#1
      Solaris
      Hudson Solaris test
      Windows #3
      35
    • 38. Concurrent Builds
      36
      Hudson #10
      Hudson #11
      Hudson #12
      Hudson #13
      Hudson #14
      Hudson #15
      time
    • 39. Making builds sticky
      We want jobs to be mostly on the same slave
      Faster check out
      Consistent results
      Minimizes disk consumption
      But does it softly
      Hudson uses consistent hash* to achieve this
      More schedule controls become possible:
      Use faster machines more frequently
      Slowly ramp up newly installed slaves
      * http://en.wikipedia.org/wiki/Consistent_hashing
      37
    • 40. Forecasting failures
      Hudson monitors key health metrics of slaves
      Low disk space, insufficient swap
      Clock out of sync
      Extensible
      Slaves go offline automatically
      38
    • 41. Clean up mess after builds
      Kill run-away processes
      Daemons, background processes left by your build
      Works on Windows, Linux, Mac, and Solaris
      39
    • 42. Load Statistics Monitoring
      40
    • 43. When it’s time to add more slaves
      41
    • 44. Hudson EC2 plugin
      Automatically provisions slaves on EC2 on demand
      Based on the current load
      Picks the right AMI depending on demand
      Starts slave agent
      Shuts down unused instances
      42
    • 45. Putting it all together
      capacity
      # of executors
      usage
      queue length
      time
      43
    • 46. Hudson “Appliance” on EC2
      Run the master in the cloud too, if you like
      Hudson on stock OpenSolaris AMI
      Data stored persistently in Elastic Block Storage
      Dynamically expandable thanks to ZFS
      Online, too
      Packaged as a wizard
      44
    • 47. And a lot more I couldn’t show
      IDE plugins
      iPhone/Android apps
      REST API and CLI
      Deployment Automation
      Hadoop, Selenium

      45
    • 48. Conclusions
      CI is here to stay
      Using lots of PCs is a key
      Hudson is very easy to get started
      Once started, there are a lot you can do
      46
    • 49. Q&A
      Kohsuke Kawaguchi
      Sun Microsystems, Inc.
    • 50. Compartmentalization
      full Metro build #10
      full Metro build #11
      JAXB #10
      JAXB #11
      JAXB #12
      Faster feedback
      Good engineering practice anyway
      Isolation of problems
      JAX-WS #20
      JAX-WS #21
      JAX-WS #22
      time
      48
    • 51. Automated System Installations
      Supports OpenSolaris, Ubuntu, CentOS, Fedora
      Trivial with most Linux
      Cooperate with Windows, too
      Quite useful outside Hudson, too
      No more broken CD drives
      No more CD-Rs
      49

    ×