Improving Engineering Process
Using Hudson
     <Insert Picture Here>




Arun Gupta, Java EE & GlassFish Guy
Oracle Corp
...
Never send a human to do a machine's job.
Throughout human history, we have been
   dependent on machines to survive.
Rise of Continuous Integration

    Offload from people, push to computers

$


      computers



       people
         ...
Spend more CPU power to help you

First on your laptops and workstations
●

     ●   IDEs are at the forefront
And then to...
Hudson (http://hudson-ci.org)

● Open-source CI server
● Emphasis on ease of installation and use

      ● “java -jar huds...
"D
     as
        hb
          oa
            rd
                 [H
                   ud
                     so
      ...
Basic Feature Set
●   Hudson notices that there is a code change
●   Checks out the source code
        ●   CVS, Subversio...
Provides Feedback
●   RSS feeds
●   Email/IM notification
●   Tray application to notify developers
     ● Linux/Windows (...
Demo
Why do I care?
Life before Hudson
●

     ● Dev forgets to commit a file
     ● Nightly build breaks, Testing blocked


  ...
Why do I care?
●   Life before Hudson
      ● Dev makes a change Monday AM
      ● Test nightly finds a bug Monday night

...
Why do I care?

Life before Hudson
●

     ● Test runs every night, results in e-mail
     ● Excitement fades after 1 week...
Matrix projects
Run the same thing on
●

 different environments
     ●   JDKs
     ●   Databases
     ●   OSes
     ●   A...
FindBugs integration
Tracking changes




        16
… and more
Browse workspaces   Build time trend report
Inter-Project Support
Host many projects on 1 Hudson
●

    ●   1+1 > 2
You can…
●

    ● Execute tests separately and cor...
Test/Build Separation
 ●Tests often dominate the build time
 ●Multiple tests are run in parallel

     ●   Improves turn-a...
Build Promotion
CI produces a lot of “successful” builds
●

     ●   Often overwhelming to downstream
          consumers
...
Got the idea?
   Automation
   Reduce turn-around time
   Make things transparent
   Remove people from the loop
   S...
Doing Distributed Builds




                           22
Distributed builds with Hudson

Master
●

     ● Serves HTTP requests                     slave
     ● Stores all importan...
Automated System Installations

          ●   Hudson + PXE plugin
                ● ISO images of OS


                   ...
Automated Tool Installation - JDK
JDK from http://java.sun.com/
●

     ● Automatically chooses the right bundle
     ● Al...
Automated Tool Installation - Apache


    Ant and Maven from Apache




                                       26
Heterogeneous Cluster Challenge
   Builds/tests need to run in specific environment
   Dependency on individual nodes hu...
Labels to rescue
   Label is a group of slaves
   Tie jobs to labels
           jobs                   slaves

         ...
Forecasting failures
Hudson monitors key health metrics of
●

 slaves
     ● Low disk space, insufficient swap
     ● Cloc...
Load Statistics Monitoring
When it’s time to add more slaves




                                    31
Hudson EC2 plugin

Automatically provisions slaves on EC2 on
●

 demand
     ●   Based on the current load
●Picks the righ...
And a lot more . . .


    IDE plugins
    iPhone/Android apps
    REST API and CLI
    Deployment Automation
    Had...
HUMANS: ALL YOUR COMPUTER ARE BELONG
TO US
Conclusions

CI is here to stay
●

     ●   Using lots of PCs is a key

Hudson is very easy to get started
●

● Once start...
Upcoming SlideShare
Loading in...5
×

Improving Engineering Processes using Hudson - Spark IT 2010

4,052

Published on

Improving Engineering Processes using Hudson - Spark IT 2010

Published in: Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,052
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
218
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Improving Engineering Processes using Hudson - Spark IT 2010

  1. 1. Improving Engineering Process Using Hudson <Insert Picture Here> Arun Gupta, Java EE & GlassFish Guy Oracle Corp http://blogs.sun.com/arungupta, @arungupta
  2. 2. Never send a human to do a machine's job.
  3. 3. Throughout human history, we have been dependent on machines to survive.
  4. 4. Rise of Continuous Integration Offload from people, push to computers $ computers people Time
  5. 5. Spend more CPU power to help you First on your laptops and workstations ● ● IDEs are at the forefront And then to the servers ● ● a.k.a. “Continuous Integration” ● More frequent build/test executions ● Static code analysis tools ● And more to come
  6. 6. Hudson (http://hudson-ci.org) ● Open-source CI server ● Emphasis on ease of installation and use ● “java -jar hudson.war” execution ● Or your OS-specific package ● Configure everything from browser ● Extensibility ● 230+ community-developed public plugins ● By 230+ contributors ● Estimated 13,000 installations
  7. 7. "D as hb oa rd [H ud so n] "
  8. 8. Basic Feature Set ● Hudson notices that there is a code change ● Checks out the source code ● CVS, Subversion, ClearCase, Mercurial, Accurev, Perforce, Git, StarTeam, … ● Builds Ant, Maven, shell script, NAnt, … ● Record and publish results ● Tool Integration ● Findbugs, Corbertura, ...
  9. 9. Provides Feedback ● RSS feeds ● Email/IM notification ● Tray application to notify developers ● Linux/Windows (Toolbar), Mac (Growl) ● ...
  10. 10. Demo
  11. 11. Why do I care? Life before Hudson ● ● Dev forgets to commit a file ● Nightly build breaks, Testing blocked ● Resources not utilized efficiently Life after Hudson ● ● Hudson will tell you in 5 mins if a build broke ● No one else notices that you forgot to commit a file ;-)
  12. 12. Why do I care? ● Life before Hudson ● Dev makes a change Monday AM ● Test nightly finds a bug Monday night ● Dev fixes it Tuesday ● Again, under utilization of resources Life after Hudson ● ● Dev makes a change Monday AM ● Hudson finds a regression few mins later ● Dev can fix it before lunch
  13. 13. Why do I care? Life before Hudson ● ● Test runs every night, results in e-mail ● Excitement fades after 1 week ● Regressions go unnoticed until it’s too late Life after Hudson ● ● Tests run Hudson after every commit ● E-mail sent out only when tests start failing ● So it manages to keep people’s attention
  14. 14. Matrix projects Run the same thing on ● different environments ● JDKs ● Databases ● OSes ● App servers ● XML parsers ● ... Results aggregated ●
  15. 15. FindBugs integration
  16. 16. Tracking changes 16
  17. 17. … and more Browse workspaces Build time trend report
  18. 18. Inter-Project Support Host many projects on 1 Hudson ● ● 1+1 > 2 You can… ● ● Execute tests separately and correlate them with builds ● Mark builds as “promoted” based on test outcome ● Track which build of libraries are used where ● . . .
  19. 19. Test/Build Separation ●Tests often dominate the build time ●Multiple tests are run in parallel ● Improves turn-around time time JAX-WS #10 JAX-WS #11 JAX-WS #12 … Unit test with GF Unit test with Tomcat SQE test Test w/o 3OS x 2JDK = 6 tests container
  20. 20. Build Promotion CI produces a lot of “successful” builds ● ● Often overwhelming to downstream consumers ●Run tests as fast as you can ●If a build passes tests, promote it 20
  21. 21. Got the idea?  Automation  Reduce turn-around time  Make things transparent  Remove people from the loop  Save people’s time  Push jobs to servers, keep workstations idle for you
  22. 22. Doing Distributed Builds 22
  23. 23. Distributed builds with Hudson Master ● ● Serves HTTP requests slave ● Stores all important info 1 slave slave Slaves ● 8 2 ● 170KB single JAR ● Assumed to be slave slave unreliable 7 Master 3 ● Scale to at least 100 Link ● slave slave 6 4 ● Single bi-di byte stream slave ● No other requirements 5
  24. 24. Automated System Installations ● Hudson + PXE plugin ● ISO images of OS Your corporate IT guy & his DHCP server ● Slaves ● Power on, hit F12 ● PC boots from network (PXE) ● Choose OS from menu ● Installs non-interactively
  25. 25. Automated Tool Installation - JDK JDK from http://java.sun.com/ ● ● Automatically chooses the right bundle ● Always up to date with new releases 25
  26. 26. Automated Tool Installation - Apache  Ant and Maven from Apache 26
  27. 27. Heterogeneous Cluster Challenge  Builds/tests need to run in specific environment  Dependency on individual nodes hurts utilization jobs slaves Wombat Windows Windows test #1 GlassFish Windows Windows test #2 Hudson Solaris Windows test #1 Hudson Solaris test
  28. 28. Labels to rescue  Label is a group of slaves  Tie jobs to labels jobs slaves Wombat Windows Windows test #1 Windows GlassFish Windows Windows test #2 Hudson Solaris Windows test #1 Solaris Hudson Windows Solaris test #3
  29. 29. Forecasting failures Hudson monitors key health metrics of ● slaves ● Low disk space, insufficient swap ● Clock out of sync ● Extensible Slaves go offline automatically ●
  30. 30. Load Statistics Monitoring
  31. 31. When it’s time to add more slaves 31
  32. 32. Hudson EC2 plugin Automatically provisions slaves on EC2 on ● demand ● Based on the current load ●Picks the right AMI depending on demand ●Starts slave agent ●Shuts down unused instances Can run Hudson master in the cloud too ●
  33. 33. And a lot more . . .  IDE plugins  iPhone/Android apps  REST API and CLI  Deployment Automation  Hadoop, Selenium  …
  34. 34. HUMANS: ALL YOUR COMPUTER ARE BELONG TO US
  35. 35. Conclusions CI is here to stay ● ● Using lots of PCs is a key Hudson is very easy to get started ● ● Once started, there is a lot you can do hudson-ci.org ●
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×