JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild
•
2 likes•265 views
By Jonathann Zenou, Ravello Systems Ravello Systems has relied heavily on Jenkins since the early days. During the company's first four years, their Jenkins setup changed and evolved to the point that it was out of control. Ravello decided to take all the experience from those years and create a new and improved Jenkins setup. This session will share the lessons that were learned the hard way - and explain how the CI process was optimized. The process of re-doing the CI mechanism for a large always developing group will be discussed, as well as mechanisms for revision control in Jenkins, job infrastructure and architecture guidelines for maximal flexibility and reuse and various other considerations. The overall continuous integration and testing strategies - which are completely cloud-based and cover all our varied components (from a hypervisor through networking and storage layers, distributed backend systems and all the way to an HTML5-based UI and a cross platform client side application) - will be described.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (19)
Similar to JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild
Similar to JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild (20)
More from CloudBees
More from CloudBees (14)
Recently uploaded
Recently uploaded (20)
JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild
- 1. Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild Europe, June 2015 Jonathann Zenou @zenoujohn
- 2. Footer #jenkinsconf Greetings fellow humans • Hi, my name is Jonathann • Living in Tel Aviv, Israel • Originally from Marseille, France • Leading the DevOps effort at Ravello Systems • jonathann.zenou@ravellosystems.com 2
- 3. #jenkinsconf Footer What is Ravello Systems ?
- 4. #jenkinsconf Footer Ravello Systems • 30 developers • 4 teams • 7 major projects • Kanban like 4
- 5. #jenkinsconf Footer A part of our stack 5
- 6. #jenkinsconf Footer What are we going to talk about ? 6
- 8. #jenkinsconf Footer Jobs Leftovers • Causes full disk • Workspace Cleanup Plugin • Can pollute your maven repo or npm cache • Solution needed : a new clean environment for every job 8
- 9. Footer #jenkinsconf Better resource distribution 9 Backend Team Frontend Team
- 10. Footer #jenkinsconf Better resource distribution 10 Backend Team Frontend Team
- 11. Footer #jenkinsconf Better resource distribution 11 Backend Team Frontend Team
- 12. Footer #jenkinsconf Better resource distribution 12 Backend Team Frontend Team
- 13. Footer #jenkinsconf Better resource distribution 13 Backend Team Frontend Team “waiting for next available executor …”
- 14. Footer #jenkinsconf Better resource distribution 14 Backend Team/Frontend Team
- 15. Footer #jenkinsconf Better resource distribution 15 Backend Team/Frontend Team
- 16. Footer #jenkinsconf Better resource distribution 16 Backend Team/Frontend Team
- 17. Footer #jenkinsconf Frontend Team leader’s reaction 17
- 18. #jenkinsconf Footer Better resource distribution • Solution needed : a rapid and scalable way to spawn dynamic environments 18
- 20. #jenkinsconf Footer Using Docker in Jenkins • Using Jenkins Docker plugin • Dedicated Docker host for each team • Hosts machine capabilities are a function of team size and requirements • All images on all hosts • Problems solved : jobs leftovers and resource distribution 20
- 21. #jenkinsconf Footer Security • No active directory or LDAP • Jim uses Dave’s account • Employee leaves the company 21
- 24. #jenkinsconf Footer Security • Solution needed: third party dynamic authentication mechanism 24
- 25. #jenkinsconf Footer Security • Using Jenkins Google Login plugin • Coupled with Matrix Authorization Strategy plugin • 2-Factor authentication • Open our Jenkins to the world 25
- 26. #jenkinsconf Footer Too much code in jobs 26
- 27. #jenkinsconf Footer Too much code in jobs • Hard to maintain • Can be easily broken • “Who touched this ??” 27
- 28. #jenkinsconf Footer Too much code in jobs • Solution: • Put your scripts in revision control (Github) • Using Jenkins Multiple SCM plugin 28
- 30. #jenkinsconf Footer Jobs Versioning • Solution: • Using Jenkins JobConfigHistory plugin • Saves old configurations • Diff between job configurations • Revert job configuration • Coupled with Jenkins SCM sync configuration plugin • Saves you whole Jenkins configuration in revision control (Github) 30
- 31. #jenkinsconf Footer Jobs data sharing 31 Backend Job Frontend Job Parameters: - Backend version: 1.1 - Frontend version: stable Parameters: - Frontend version: 1.1 - Backend version: stable
- 32. #jenkinsconf Footer Jobs data sharing • Solution needed: an interface/service that will store data that will be accessible from all the jobs • Using Jenkins Global Variable String Parameter Plugin • Cannot alter them from a job • But you can using Groovy … 32
- 34. #jenkinsconf Footer Jobs data sharing • Keep a global parameter for each of product stable version • Let all the other jobs use these values • Change the global parameter value of a product stable version automatically when this product passes a green CI • Gist link: https://gist.github.com/johnyzed/2af71090419af2b20c5a 34
- 35. #jenkinsconf Footer CI at Ravello Systems 35 build + unit tests environment creation code deployment test suites stop environments Artifactory log collection
- 36. #jenkinsconf Footer CI at Ravello Systems 36 environment creation
- 38. #jenkinsconf Footer The pieces are ready • What Do Developers Want ? • Blackbox • No duplication • C.D.R.S • Single form, with a lot of triggers • Using Multijob Plugin 38
- 39. #jenkinsconf Footer Jobs sandbox • Improving a job without: • Breaking it • Stopping other people using it • Solution needed: a parallel Jenkins environment almost identical to our production Jenkins environment to test the new changes on 39
- 40. #jenkinsconf Footer Jobs sandbox • Jenkins environment blueprint on Ravello Systems • Using Ansible: • git clone • minor changes • Running regular jobs • All that orchestrated from a Jenkins job 40
- 41. #jenkinsconf Footer Jenkins sandbox flow 41 changes on Jenkins script branch create dynamic Jenkins environment run jobs stop dynamic environment
- 43. #jenkinsconf Footer Comparing Jenkins environment 43 Old Jenkins New Jenkins Number of jobs 137 56 Number of slaves 10 0-50 Build and test duration 70 43
- 45. #jenkinsconf Footer Take-aways • Ask your developers how to make their life easier • Don’t be afraid to break things • Culture • Follow @jenkins_release • Follow @devopsreactions 45
- 47. #jenkinsconf Footer One last thing • Ravello Repo • http://www.ravellosystems.com/repo/ • http://www.ravellosystems.com/repo/blueprints/60391436 47
- 49. #jenkinsconf Footer Please Share Your Feedback • Did you find this session valuable? • Please share your thoughts in the Jenkins User Conference Mobile App. • Find the session in the app and click on the feedback area. 49