Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Jenkins User Conference   Paris, 17 April 2012   #jenkinsconf                 Julien Carsique, DevOps                 http...
Jenkins User Conference              Paris, 17 April 2012   #jenkinsconf    The Nuxeo company — www.nuxeo.com     ●       ...
Jenkins User Conference       Paris, 17 April 2012   #jenkinsconf    The Nuxeo products    Document and Case Management17/...
Jenkins User Conference         Paris, 17 April 2012   #jenkinsconf    The Nuxeo products    Digital Assets Management17/0...
Jenkins User Conference   Paris, 17 April 2012   #jenkinsconf    The Nuxeo products    Web sites17/04/2012                ...
Jenkins User Conference            Paris, 17 April 2012   #jenkinsconf    The Nuxeo products    Public facing applications...
Jenkins User Conference                  Paris, 17 April 2012   #jenkinsconf    The Nuxeo products    Open Source ECM plat...
Jenkins User Conference            Paris, 17 April 2012   #jenkinsconf    The challenge of    The source code     ●       ...
Jenkins User Conference                        Paris, 17 April 2012   #jenkinsconf    The challenge of    The human constr...
Jenkins User Conference                     Paris, 17 April 2012   #jenkinsconf    The challenge of    The deployment cons...
Jenkins User Conference                             Paris, 17 April 2012   #jenkinsconf    The history of    The beginning...
Jenkins User Conference                                                 Paris, 17 April 2012            #jenkinsconf    Th...
Jenkins User Conference                        Paris, 17 April 2012   #jenkinsconf    The history of    Nuxeo 5.1 — CI arc...
Jenkins User Conference                           Paris, 17 April 2012   #jenkinsconf    The history of    Nuxeo 5.1 to 5....
Jenkins User Conference                 Paris, 17 April 2012   #jenkinsconf    The history of    Nuxeo 5.3 to 5.4 (2010-20...
Jenkins User Conference                   Paris, 17 April 2012   #jenkinsconf    The history of    Nuxeo 5.4 (2011) — CI i...
Jenkins User Conference                         Paris, 17 April 2012     #jenkinsconf    The history of    Nuxeo 5.4 (2011...
Jenkins User Conference              Paris, 17 April 2012   #jenkinsconf    The history of    Nuxeo 5.5 — the current stat...
Jenkins User Conference                              Paris, 17 April 2012   #jenkinsconf    The history of    Nuxeo 5.5 — ...
Jenkins User Conference                                                  Paris, 17 April 2012                            #...
Jenkins User Conference                           Paris, 17 April 2012         #jenkinsconf    The history of    Nuxeo 5.5...
Jenkins User Conference                 Paris, 17 April 2012   #jenkinsconf    The history of    Nuxeo 5.6 — the near futu...
Jenkins User Conference                  Paris, 17 April 2012   #jenkinsconf    The history of    Nuxeo 5.6 — the near fut...
Jenkins User Conference                       Paris, 17 April 2012   #jenkinsconf    Conclusion     ●         Jenkins     ...
2517/04/2012    Jenkins User Conference Paris   17 April 2012   #jenkinsconf        Thank You To Our Sponsors        Plati...
Upcoming SlideShare
Loading in …5
×

How Nuxeo uses the open-source continuous integration server Jenkins

4,926 views

Published on

Learn how Nuxeo has been using Jenkins for the past five years.

Published in: Technology
  • Be the first to comment

How Nuxeo uses the open-source continuous integration server Jenkins

  1. 1. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf Julien Carsique, DevOps https://qa.nuxeo.org/jenkins
  2. 2. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The Nuxeo company — www.nuxeo.com ● Founded in 2000 ● ~ 50 employees (mainly France & USA) ● Open Source Enterprise Content Management ● No-charge software licensing with LGPL ● Nuxeo Connect subscription offering for support, maintenance, consulting, training and services: ● Nuxeo Studio ● Nuxeo Marketplace ● Nuxeo Cloud17/04/2012 2/25
  3. 3. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The Nuxeo products Document and Case Management17/04/2012 3/25
  4. 4. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The Nuxeo products Digital Assets Management17/04/2012 4/25
  5. 5. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The Nuxeo products Web sites17/04/2012 5/25
  6. 6. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The Nuxeo products Public facing applications, RIA and much more …17/04/2012 6/25
  7. 7. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The Nuxeo products Open Source ECM platform, applications and services ● Nuxeo Platform ● Java and standards based ● Component based architecture ● Extensible and configurable ECM services ● UI building blocks and High level frameworks ● A lot of integration levels and APIs ● Testing, benchmarking and packaging tools ● Development tools (Nuxeo Studio, Nuxeo IDE) ● Ready to use pre-packaged applications17/04/2012 7/25
  8. 8. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The challenge of The source code ● 200+ SubVersion, Mercurial and Git repositories ● 1M+ LoC, 500+ Maven modules ● 5 application modules and 60+ addons ● 500+ components in the Platform ● 2500+ unit tests, 2000+ Selenium and WebDriver tests ● 10+ languages (Java, XML, JavaScript, HTML, CSS, Python, Shell, SQL, Groovy, …) and tools (Maven, Ant, Shell, …) ● 30~50 active committers and contributors ● ~1000 commits/month ● 1 development branch, 3~5 maintenance branches17/04/2012 8/25
  9. 9. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The challenge of The human constraints ● Follow and maintain quality ● Save developers time ● Doing more checks than they can ● Providing a safety net for their changes ● No need to build everything themselves ● Save testers time ● Automate tests in various environments ● Warn on regression ● Easy to download latest deliverables ● Save integrators time ● Work on Nuxeo development branch with more confidence ● Get guarantees on code quality17/04/2012 9/25
  10. 10. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The challenge of The deployment constraints ● Multiple deployment target ● WAR, JEE or bare Java ● OS (Linux, Unix, Windows, MacOS) ● DB (PostgreSQL, Oracle, SQLServer, MySQL, …) ● BlobStore backend (FS, NAS, S3, SQLDB, ...) ● Scalability and performance ● Cluster support ● Multi-DB scale-out (several repositories and/or DB) ● Raw performances (100M docs benchmark on a single server; 200 pages/s output on complex business apps) ● Easy maintenance ● Backup, restore, upgrade ● export/import17/04/2012 10/25
  11. 11. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The history of The beginning ● Before 2007: BuildBot ● 2007 — Nuxeo 5 switch to Java ● Trying CruiseControl, TeamCity, Continuum, … Hudson! ● Goals for Nuxeo 5.1 ● Code analysis (Quality Assurance) ● Build and unit test Nuxeo against default environment – Linux, JBoss, Sun JDK 5, H2 – Smoke test at JBoss start – Run Selenium suite ● Automate Nuxeo release process ● Automate generation and test of documentation ● Involve developers17/04/2012 11/25
  12. 12. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The history of Nuxeo 5.1 — CI architectureRELEASES and SNAPSHOTS Maven.nuxeo.org from outside RE UPLOAD RELEASES & SNAPSHOTS RE S LEA LE SE SE S AS EA L ES RE Public Internal Nexus Private Internal Nexus Nexus Preprod DOWNLOAD DOWNLOAD UPLOAD UPLOAD DOWNLOAD UPLOADPublic Jenkins Master Private Jenkins Master17/04/2012 12/25
  13. 13. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The history of Nuxeo 5.1 — CI architecture and usage ● Speed up build time ● Organize teams resources ● Ensure publicly available artifacts quality ● Isolate internal and external build chains ● Enforce rules for developers ● Early use of CI ● Think about what kind of build (Maven, Shell, Ant, …) and what must be checked for validation (unit tests, errors in logs, …)? ● Automated build: nothing specific or contextual to the slave ● Make the build self-testing ● Stay tuned, be aware of CI activity ● Never let a project in an unstable status17/04/2012 13/25
  14. 14. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The history of Nuxeo 5.1 to 5.3 — making heavy use of CI ● Release process improvement ● Automated tasks ● Nightly / On demand ● Build promotion ● More and more tests and environments ● More performance and functional tests ● Tests with Jetty, JBoss and Tomcat ● Matrix testing jobs with OS and databases ● Testing development and administration tools ● Seriously increasing number of builds ● New packages and installers (Debian, VM, Windows, Multi-OS, …) ● New applications (Android, Nuxeo CMF, Nuxeo DAM, …) ● New plugins for Firefox, IE, OpenOffice/LibreOffice ● Accumulating versions to maintain17/04/2012 14/25
  15. 15. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The history of Nuxeo 5.3 to 5.4 (2010-2011) — growing CI issues ● CI performance issues ● CPU, network and disk usage ● Slave usage strategy ● Full cycle duration ● CI usage issues ● Build waiting time ● Feedback delay ● Feedback accuracy ● Jobs dependencies17/04/2012 15/25
  16. 16. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The history of Nuxeo 5.4 (2011) — CI improvements, Jenkins analysis ● Build cycles, workflows and durations ● Use of Jenkviz (homemade tool), Dependency Graph View and (enhanced) Build Timeout plugins17/04/2012 16/25
  17. 17. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The history of Nuxeo 5.4 (2011) — CI improvements, solutions ● Work on source code ● GWT optimizations (reduce build time) ● Review testing suites, prefer use of WebDriver over Selenium ● Work on CI ● Improve slaves usage (using labels), local vs remote ● Give priority to jobs with low duration (earlier feedback) ● New short and permanent cycle ● Create independent jobs for specific purposes ● Create new views to help developers ● Add CPU and/or memory on some slaves ● Improve Nexus infrastructure (Maven build time) ● Fix jobs triggering issue17/04/2012 17/25
  18. 18. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The history of Nuxeo 5.5 — the current status17/04/2012 18/25
  19. 19. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The history of Nuxeo 5.5 — the current status ● Great coverage and good delays ● 4 kinds of CI cycles per Nuxeo version maintained – Triggered by commit (300 jobs) – Full build and tests per day (10 jobs) – Independent cycle per 3 hours (11 jobs) – Nightly build (40 jobs) ● Less than 10 minutes to 3 hours max for main feedback ● Code and applications tested on multi-DB/OS/JDK/servers ● 6 hours for the longest build cycle ● About 450 jobs on 25 servers (2 masters), 200~?K builds/day ● 6 database engines tested on 5 operating systems ● 7 applications tested in all environments ● 50K~100K tests ran including all products and versions under CI17/04/2012 19/25
  20. 20. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The history of Nuxeo 5.5 — the current status, major CI cycles Job trigger Performance tests multi-DB Commit SCM Sources Build & Packaging Functional tests multi-OS/DBMercurial/GitHub trigger checkout Unit tests 7 apps, 2 servers Various addons functional tests Full build & tests multi-JDK daily Sources Packaging multi-OS Smoke tests trigger Full build & tests multi-DB Linux checkout Full build & tests multi-DB Win 3h Sources Merge « stable » trigger Full build (no tests) Packaging all applications Functional tests checkout branch (GitHub) Permanent cycle Nightly Sources Full build (no tests) Functional tests Packaging all applications multi-DB/OS/JDK/servers, cluster Manual checkout Prepare release Batch task Performance tests (référence) Perform release Build installers (Debian, Merge «5.6-SNAPSHOT» Win, multi-OS, ...) branch (GitHub) Push release tag (GitHub) Deploy «5.6-SNAPSHOT» Test and upload installers Deploy release artifacts Maven artifacts17/04/2012 20/25
  21. 21. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The history of Nuxeo 5.5 — main Jenkins plugins used ● Required ● Subversion, Mercurial, MSBuild, GIT, GitHub, Android Emulator ● Useful for Jenkins management ● Priority Sorter, Parameterized Trigger, Retry Failed Builds, Rebuilder, Hudson Post build task, Matrix Reloaded, next-executions, Bulk Builder, Join, Promoted Builds, batch task ● Change Log History, Maven Deployment Linker, SCM Sync Configuration, Multiple SCMs, Static Analysis Utilities, Timestamper, build timeout, Hudson Setenv, java.io.tmpdir cleaner, Hudson global-build-stats, Node and Label parameter, Monitoring, SSH Slaves, Dependency Graph Viewer, Performance, Disk Usage ● Useful reports ● Checkstyle, FindBugs, Task Scanner, PMD, Warnings, Hudson Seleniumhq, Selenium HTML report, Cobertura, TextFinder, JUnit Attachments, Duplicate Code Scanner ● For end-users ● Dashboard View, Claim, Radiator View, Hudson iPhoneView, Continuous Integration game, instant-messaging, Jabber notifier, built-on-column, Compact Columns, Version Column, Email-ext, View Job Filters, Extra Columns, Simple Theme17/04/2012 21/25
  22. 22. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The history of Nuxeo 5.6 — the near future ● Better quality testing with Sonar ● Integration with Jira ● Automated merges from QA dedicated branches to the development branch ● Automated Jenkins infrastructure deployment on cloud ● Custom slaves on demand (Chef / Puppet, chroot / LXC, …) ● Work on Nuxeo Cloud & Compatible One R&D ● CloudBees ● Nuxeo CI (partial or full) migration to the cloud17/04/2012 22/25
  23. 23. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf The history of Nuxeo 5.6 — the near future ● Jenkins enhancements (plugins) ● More abstraction to avoid plugins compliance ● Multiple SCM & Git ● SCM change log feeder ● Matrix jobs improvements ● Node Label & Parameterized trigger ● Dependency Graph ● Infrastructure failures (Check job prerequisites++) ● Jobs workflow management ● …17/04/2012 23/25
  24. 24. Jenkins User Conference Paris, 17 April 2012 #jenkinsconf Conclusion ● Jenkins ● Easy maintenance ● Good scalability ● Great community and plugins catalog ● Powerful extensibility ● Efficient contribution process ● Cloud capabilities ● Nuxeo ● CI & QA on Nuxeo Platform frameworks ● Stable APIs with backward and forward compatibility ● Stable applications with good performances ● Automated processes with very few manual interventions17/04/2012 24/25
  25. 25. 2517/04/2012 Jenkins User Conference Paris 17 April 2012 #jenkinsconf Thank You To Our Sponsors Platinum Sponsor Gold Sponsor Silver Sponsor

×