Improving Engineering Processes using Hudson - Spark IT 2010
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Improving Engineering Processes using Hudson - Spark IT 2010

  • 5,368 views
Uploaded on

Improving Engineering Processes using Hudson - Spark IT 2010

Improving Engineering Processes using Hudson - Spark IT 2010

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
5,368
On Slideshare
5,094
From Embeds
274
Number of Embeds
11

Actions

Shares
Downloads
215
Comments
0
Likes
6

Embeds 274

https://blogs.oracle.com 85
http://blogs.oracle.com 74
http://blogs.sun.com 49
http://blog.arungupta.me 34
http://www.slideshare.net 13
http://www.techgig.com 8
http://cache.baidu.com 4
http://planets.sun.com 3
http://translate.googleusercontent.com 2
http://webcache.googleusercontent.com 1
http://timesjobs.techgig.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Improving Engineering Process Using Hudson <Insert Picture Here> Arun Gupta, Java EE & GlassFish Guy Oracle Corp http://blogs.sun.com/arungupta, @arungupta
  • 2. Never send a human to do a machine's job.
  • 3. Throughout human history, we have been dependent on machines to survive.
  • 4. Rise of Continuous Integration Offload from people, push to computers $ computers people Time
  • 5. Spend more CPU power to help you First on your laptops and workstations ● ● IDEs are at the forefront And then to the servers ● ● a.k.a. “Continuous Integration” ● More frequent build/test executions ● Static code analysis tools ● And more to come
  • 6. Hudson (http://hudson-ci.org) ● Open-source CI server ● Emphasis on ease of installation and use ● “java -jar hudson.war” execution ● Or your OS-specific package ● Configure everything from browser ● Extensibility ● 230+ community-developed public plugins ● By 230+ contributors ● Estimated 13,000 installations
  • 7. "D as hb oa rd [H ud so n] "
  • 8. Basic Feature Set ● Hudson notices that there is a code change ● Checks out the source code ● CVS, Subversion, ClearCase, Mercurial, Accurev, Perforce, Git, StarTeam, … ● Builds Ant, Maven, shell script, NAnt, … ● Record and publish results ● Tool Integration ● Findbugs, Corbertura, ...
  • 9. Provides Feedback ● RSS feeds ● Email/IM notification ● Tray application to notify developers ● Linux/Windows (Toolbar), Mac (Growl) ● ...
  • 10. Demo
  • 11. Why do I care? Life before Hudson ● ● Dev forgets to commit a file ● Nightly build breaks, Testing blocked ● Resources not utilized efficiently Life after Hudson ● ● Hudson will tell you in 5 mins if a build broke ● No one else notices that you forgot to commit a file ;-)
  • 12. Why do I care? ● Life before Hudson ● Dev makes a change Monday AM ● Test nightly finds a bug Monday night ● Dev fixes it Tuesday ● Again, under utilization of resources Life after Hudson ● ● Dev makes a change Monday AM ● Hudson finds a regression few mins later ● Dev can fix it before lunch
  • 13. Why do I care? Life before Hudson ● ● Test runs every night, results in e-mail ● Excitement fades after 1 week ● Regressions go unnoticed until it’s too late Life after Hudson ● ● Tests run Hudson after every commit ● E-mail sent out only when tests start failing ● So it manages to keep people’s attention
  • 14. Matrix projects Run the same thing on ● different environments ● JDKs ● Databases ● OSes ● App servers ● XML parsers ● ... Results aggregated ●
  • 15. FindBugs integration
  • 16. Tracking changes 16
  • 17. … and more Browse workspaces Build time trend report
  • 18. Inter-Project Support Host many projects on 1 Hudson ● ● 1+1 > 2 You can… ● ● Execute tests separately and correlate them with builds ● Mark builds as “promoted” based on test outcome ● Track which build of libraries are used where ● . . .
  • 19. Test/Build Separation ●Tests often dominate the build time ●Multiple tests are run in parallel ● Improves turn-around time time JAX-WS #10 JAX-WS #11 JAX-WS #12 … Unit test with GF Unit test with Tomcat SQE test Test w/o 3OS x 2JDK = 6 tests container
  • 20. Build Promotion CI produces a lot of “successful” builds ● ● Often overwhelming to downstream consumers ●Run tests as fast as you can ●If a build passes tests, promote it 20
  • 21. Got the idea?  Automation  Reduce turn-around time  Make things transparent  Remove people from the loop  Save people’s time  Push jobs to servers, keep workstations idle for you
  • 22. Doing Distributed Builds 22
  • 23. Distributed builds with Hudson Master ● ● Serves HTTP requests slave ● Stores all important info 1 slave slave Slaves ● 8 2 ● 170KB single JAR ● Assumed to be slave slave unreliable 7 Master 3 ● Scale to at least 100 Link ● slave slave 6 4 ● Single bi-di byte stream slave ● No other requirements 5
  • 24. Automated System Installations ● Hudson + PXE plugin ● ISO images of OS Your corporate IT guy & his DHCP server ● Slaves ● Power on, hit F12 ● PC boots from network (PXE) ● Choose OS from menu ● Installs non-interactively
  • 25. Automated Tool Installation - JDK JDK from http://java.sun.com/ ● ● Automatically chooses the right bundle ● Always up to date with new releases 25
  • 26. Automated Tool Installation - Apache  Ant and Maven from Apache 26
  • 27. Heterogeneous Cluster Challenge  Builds/tests need to run in specific environment  Dependency on individual nodes hurts utilization jobs slaves Wombat Windows Windows test #1 GlassFish Windows Windows test #2 Hudson Solaris Windows test #1 Hudson Solaris test
  • 28. Labels to rescue  Label is a group of slaves  Tie jobs to labels jobs slaves Wombat Windows Windows test #1 Windows GlassFish Windows Windows test #2 Hudson Solaris Windows test #1 Solaris Hudson Windows Solaris test #3
  • 29. Forecasting failures Hudson monitors key health metrics of ● slaves ● Low disk space, insufficient swap ● Clock out of sync ● Extensible Slaves go offline automatically ●
  • 30. Load Statistics Monitoring
  • 31. When it’s time to add more slaves 31
  • 32. Hudson EC2 plugin Automatically provisions slaves on EC2 on ● demand ● Based on the current load ●Picks the right AMI depending on demand ●Starts slave agent ●Shuts down unused instances Can run Hudson master in the cloud too ●
  • 33. And a lot more . . .  IDE plugins  iPhone/Android apps  REST API and CLI  Deployment Automation  Hadoop, Selenium  …
  • 34. HUMANS: ALL YOUR COMPUTER ARE BELONG TO US
  • 35. Conclusions CI is here to stay ● ● Using lots of PCs is a key Hudson is very easy to get started ● ● Once started, there is a lot you can do hudson-ci.org ●