This document provides an overview of Jenkins, an open source tool for continuous integration. It can be used to continuously build software projects and run tests whenever code changes are committed. Jenkins runs builds, tests and publishes results. It also integrates with various source control systems and build tools. The document discusses how Jenkins can be used to distribute builds across multiple computers and automate tasks like installing tools and running builds on slave nodes. It also covers advanced features like matrix projects, labels, build promotion and integration with cloud platforms.

1. Jenkins

2. What’s Jenkins?©2010 CloudBees, Inc. All Rights Reserved

3. What’s Jenkins?Jenkins is a “continuous integration” serverIt looks for changes and builds your projectIt publishes the resultIt keeps track of how your project is used elsewhereIt brings transparency to projects3

4. What’s Jenkins?Community-driven open-source projecthttp://jenkins-ci.org/Used to be called HudsonGet going with “java -jar jenkins.war”Extensible architectureLots of community developed pluginsPublic plugins as well as in-house pluginsVery active7 years old, ~400 releases to date300+ plugins, 250+ committers4

5. Used Everywhere55

6. What does Jenkins do?Notice changes and checking outPolling, post-commit hooksClearCase, Perforce, Git, Mercurial, …Runs builds and testsLots of build/reporting tools integrations6

7. What does Jenkins do?Whatever you can script can be run in JenkinsIt needs to be non-interactiveVery flexible and can work with any build/test tools/frameworksBut it is not by itself a build/test framework7

8. Non-Java SuportJenkins is used to build non-Java projectsRuby, Python, PHP, C++, .NET, …Rapidly expanding as many of those languages do not have viable CI serverVarious tool/report integrations done by community8

9. Demo©2010 CloudBees, Inc. All Rights Reserved

10. Why do I care?Life before JenkinsRE nightly build breaks. It blocks SQEDev rebuilds before commit to avoid thisPeople waste timeLife after JenkinsDev just commits it right awayDev moves on to work on another thingJenkins will tell you in 5 mins if a build broke10

11. Why do I care?Life before JenkinsDev makes a change Monday AMSQE nightly finds a bug Monday nightDev fixes it TuesdayLot of wasted timeLife after JenkinsDev makes a change Monday AMJenkins finds a regression 30 mins laterDev can fix it before lunch11

12. Why do I care?Life before JenkinsSQE test runs every night, results sent out in e-mailAfter the enthusiasm of the 1st week is gone, nobody looks at them anymoreRegressions go unnoticed until it’s too lateLife after JenkinsTests run Jenkins after every commitE-mail sent out only when tests start failingSo it manages to keep people’s attention12

13. Got the idea?AutomationReduce turn-around timeMake things transparentRemove people from the loopSave people’s timePush jobs to servers, keep workstations idle for you13

14. Advanced Integration Techniques©2010 CloudBees, Inc. All Rights Reserved

15. Doing Distributed Builds15Photo by skreuzer

16. Why?You need to use multiple computers because…You need different environmentsYou need isolationYou have non-trivial work loadsThere’s only so much you can do with 1 computer

17. Lots of computers = tediousPains grow as your build farm growsSoftware installation problemNode failure problemRemote maintenance problem…What does Jenkins do to help you with these?

18. Installing new slavesFor first 20 or so slaves, we did it manuallyInsert CD, click, type, click, type, click, …But that doesn’t scaleThen we automatedAvailable as “Jenkins PXE Plugin”

19. Automated System InstallationsJenkins + PXEplugin

20. ISO images of OSYour corporate IT guy & his DHCP serverSlavesPower on, hit F12PC boots from network (PXE)

21. Automated System InstallationsJenkins+ PXEplugin

22. ISO images of OSYour corporate IT guy & his DHCP serverSlavesPower on, hit F12PC boots from network (PXE)Choose OS from menuInstalls non-interactively

23. Automated System InstallationsSupports OpenSolaris, Ubuntu, CentOS, FedoraTrivial with most LinuxCooperate with Windows, tooQuite useful outside Jenkins, tooNo more broken CD drivesNo more CD-Rs

24. MasterServes HTTP requestsStores all important infoSlaves170KB single jarAssumed to be unreliableScale to at least 100Distributed builds with Jenkins

25. Building on slavesSlave agent is a little Java programRuns locally on the machine that it controlsAccess files, fork processes, etc., on behalf of masterCommunication with masterNeeds a bi-di byte streamBut very flexible in how this gets set upNo shared file system, no network topology constraints, etc

26. How master and slaves start talkingFor Unix slaves, via SSHOnly need SSHD on slavesWe even install Java for youWe just need a host name

27. How master and slaves start talkingFor Windows, DCOMWe just need admin user name and passwordNo manual interventionWorks even from Unix masters

28. How master and slaves start talkingVia Java Web StartWhen master cannot see slavesA separate socket connection is made

29. Automating JNLP launchOnce started, can be installed as Windows service

30. Automating JNLP launchEmulate the JNLP client headlessly$ java -jar slave.jar -jnlpUrl URL

31. Automated Tool InstallationsJDK from http://java.sun.com/It automatically chooses the right bundle

32. Automated Tool InstallationsAnt and Maven from ApacheGeneral purpose implementationsGrab an archive from some URL and extractRun arbitrary shell scriptExtensible

33. LabelsTags that group slaves togetherName: alphaLabel: windows 32bitName: bravoLabel: linux 32bitName: bravoLabel: linux 64bit

34. LabelsName: alphaLabel: windows 32bitName: bravoLabel: linux 32bitThis job needs to run on “linux”Name: bravoLabel: linux 64bit

35. LabelsName: alphaLabel: windows 32bitName: bravoLabel: linux 32bitThis job needs to run on “linux && 32bit”Name: bravoLabel: linux 64bit

36. More Label BenefitsLabels can be anythingGeographic locationsAvailability of proprietary software installedBetter resource utilizationJenkins have maximum freedomResilient to node addition/removalTreat servers like livestocks, not like pets

37. Making builds stickyWe want jobs to be mostly on the same slaveFaster check outConsistent resultsMinimizes disk consumptionBut do it softlyJenkins uses consistent hash to achieve thisMore schedule controls become possible:Use faster machines more frequentlySlowly ramp up newly installed slaves* http://en.wikipedia.org/wiki/Consistent_hashing

38. Forecasting failuresJenkins monitors key health metrics of slavesLow disk space, insufficient swapClock out of syncExtensibleSlaves put offline automaticallyCatch problems before it breaks builds36

39. Clean up mess after buildsKill run-away processesDaemons, background processes left by your buildWorks on Windows, Linux, Mac, and Solaris

40. Cloud supportSlaves can come and goCurrent/future workload and queue are all monitored continuouslyWe can let Jenkins provision slaves on cloud on-the-fly

41. Jenkins EC2 pluginAutomatically provision slaves on EC2 on demandPick the right AMI depending on demandStart slave agentShut down unused instances* http://code.google.com/p/typica/

42. Integration with VMWareSimilar mechanism exists with VMWareCreate a pool of virtual machines upfrontJenkins powers them up/down based on loadOptionally revert to snapshot at the endAchieves benefits similar to EC2 without going publicNectarFeature

43. Matrix ProjectPhoto: http://www.flickr.com/photos/amarilloposters/4380080793/41

44. MotivationYou often do the same thing with slight variationsCompile C++ code for different platformsTest with different browsersOr more generally, think of it asfor (x in [a,b,c]) { for (y in [d,e,f]) { …doSomethingWith(x,y,z,..); … }}

45. ModelDefine axesOne axis ≈ one for loopChoose from pre-defined types of AxisSlave axis: pick slaves by their names or their labelse.g., linux, solaris, and windowsGeneric axis: arbitrary values exposed as environment variables

46. Matrix Project GimmicksFilteringOtherwise combinations increase exponentiallyNot all combinations make senseUse boolean expression to trim down the sizeOr tell Jenkins to cut the workload to N%Jenkins will thin out the combinations by itself(label==“windows”).implies(browser==“iexplore”) &&(label==“mac”).implies(browser==“safari”)

47. Demo

48. Build Promotion

49. Progressive ExposureAKA Build pipelineOnce built, bits go through progressively more expensive “certification”Think of tests, deployment to QA, staging, etc.Some builds fail in the middle, others go through

50. Then it gets more complexEach stage can be several parallel jobsNeeds to be retryableYou can’t afford to execute every pipeline until it fails

51. Hence “build promotion”Configuration“Foo gets promoted if FooTest passes”“When Foo gets promoted, do X, Y, and Z”Sequence of eventsFoo #N successfully completesFooTest #M executes on Foo #NFooTest #M successfully completesFoo #N gets promotedX, Y, and Z happens on Foo #N

52. More on build promotionPromotion condition can be more complexIf FooTest, BarTest, and ZotTest all passesIf PHB manually approvesIf upstream project gets promotedExtensiblePromotion actions can be anythingCopy files, execute scripts, trigger jobs, …

53. More on build promotionMultiple different promotion levelsFoo #5FooIntTest #9FooIntTest #10FooTest #10BarTest #16Deploy #19

54. Promotion benefitsNo rebuilding of binariesAsynchronous and naturally throttledAudit Trail

55. Demo