Jenkins 1

1,392 views
1,240 views

Published on

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,392
On SlideShare
0
From Embeds
0
Number of Embeds
23
Actions
Shares
0
Downloads
37
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Jenkins 1

  1. 1. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Case Study Elad Hirsch, Evgeny Borisov Israel Direct Insurance (IDI) www.555.co.il
  2. 2. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf IDI Company products Internet site Mobile CRM BackOffice Telephonia Personal Mail Infrastructure (different ways deliveries to client)
  3. 3. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf What do we have in IDI Teams – 114 IT Teams – 12 IT employees – 121 Computers – 1200 VMs - 280 Server computers – 70 JBoss– 68 Tomcat – 14 Jetty – 8 (only for development)
  4. 4. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf What do we have in IDI 8 databases 1528 tables only in production 3 Brands (555, 9000000, Ynet) 262 wicket pages 3 development branches – Trunk – Release-Candidate – Production
  5. 5. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf IDI development software Java Java Script Delphi JAM SQR – Structured Query Reporter Store Procedures Web Services
  6. 6. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Today IDI uses… Maven Subversion Artifactory Jenkins Jira Terracotta Spring Guice EJB Hibernate
  7. 7. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Software structure Modules – 45 Classes – 28628 Methods – 200396 Code Lines – 2,147,100 155 – third party libraries 599 – third party JARs
  8. 8. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf IDI Activity 2200 Requests to JBoss per minute 2000 Emails from our clients per hour Emails to our clients per hour 303 Database transactions per second 9*4 Versioned deployments per year 216 Emergency deployments per year 160 Commits per week
  9. 9. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf 6 or 7 Years ago… Development process: – Write code than commit to SVN – Build JAR locally than commit JAR – Build EAR locally from all committed JARs – Deploy EAR
  10. 10. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Problems No synchronization between sources No synchronization between sources and JARs Runtime Exceptions as a result of common JARs change.
  11. 11. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf IDI Continues integration 'working copy'
  12. 12. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Testing and JIRA problems QA Teams can’t know what code currently deployed. Dev team leader not sure if JIRA issues were commit / deployed. What about failed commits?
  13. 13. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Analyzes SVN log information Last deployment date Find the proper modules Validate if JIRA issue is relevant to current deployment Automate the procedure !!!
  14. 14. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf JIRA & Jenkins Integration
  15. 15. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf JIRA & SVN integration
  16. 16. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Check out the Jenkins build log to keep track of Jira issues
  17. 17. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf "Real-time linking between your source code repository and JIRA" • JIRA Fisheye
  18. 18. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf IDI Continues integration 'working copy'
  19. 19. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Maintain a Single Source Repository • Subversion as Source Code Management solution. Dev/Staging/Production ENV’s.
  20. 20. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf How we can check the code before commits ? • Developers performs commits without the relevant JIRA issue. • Commits are not informative enough. • Packages are created with First Capital letter
  21. 21. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Validate that commit message in format IDI-xxxxx
  22. 22. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Maintain a Single Source Repository • Subversion as Source Code Management solution. • Dev/Staging/Production ENV’s. • Everything you need to do a build: – Source code. – Test scripts. – Properties files. – Install scripts.
  23. 23. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf What about third party libraries ? • Where should we place our JARs? In svn? • Выяснить почему неправильно хранить их вместе. • Вставить смешную картинку
  24. 24. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf IDI Continues integration 'working copy'
  25. 25. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Take control on your local and remote artifacts • Developers cannot decide to add arbitrary new versions of 3rd parties artifacts such ass Spring or RestEasy. • Time and quantity of download artifact can scale up very fast. • Control how your developers manage and download your own Enterprise artifacts.
  26. 26. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Artifactory • Optimize artifact management between developer workstation and the internet repositories. Repository Managers hide several public and internal repository URLs behind a single point of access.
  27. 27. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf
  28. 28. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf IDI Continues integration 'working copy'
  29. 29. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf User based build problems • Build takes a lot of time. • As you seen we have many commits every day. • Wasting time for every build kill us.
  30. 30. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Broken build dilemma
  31. 31. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Fast build • Take the burden of full life-cycle build out of the developer machine • First notification point for: – complex compilation problems. – Code inspection. – Modules dependencies issues. • Build that is triggered by SCM change and does an incremental compile • Tests are not executed. • Much faster then Full build. • Build artifacts don’t deployed to artifactory.
  32. 32. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Full build • Triggered nightly and builds from an empty directory, fetching all the source code prior to building. • Tests are executed. • Deploy artifacts to Artifactory repository. • In the event of failure immediate attention needed to fix the problem ASAP. • Much slower then Fast build.
  33. 33. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Build jobs – in general – Run predefine validations. Build failure
  34. 34. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Build jobs – in general – Run maven clean archives. – Install dependencyManagement artifact. – Run maven lifecycle on a refactoring POM which includes all modules. – Archive deployables -for use in Remote Test Servers (HEAD,RELEASE...) – Run some 3rd party plugins such as: • SCP plugin. • Jira integration. • Code inspection. • Continues integration game.
  35. 35. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Full Build problem • What about case, when full build failed •
  36. 36. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Selective Build • Function as Full build for specific modules • Deploy mode enable the developer to create the specific modules with the Archive deployables (EAR, WAR , Webstart…)
  37. 37. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf parameterized Jenkins Job Mojo plugin
  38. 38. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Use real Java annotations instead of JavaDocs when writing Maven plugins. Add ENV info to Astro EAR Deploy to server and monitor deployment
  39. 39. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf About 40 deployment permutations Complex deployment
  40. 40. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Narrows Deployer Server
  41. 41. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Once upon a time …. • Hudson didn’t support parameterized build. • We needed UI for complex deployment ENV’s. • Functions as deployment dashboard. • Developed new server (“Narrows”): – Wicket based. – Installed on Tomcat server. – Allot of maintenance work (configure new ENV, set up a new maven plugin , etc..) – Not our main development concern.
  42. 42. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Moving toward the jenkins • By using parameterized build and new jenkins plugins we were able to move all our Narrow based jobs into Jenkins.
  43. 43. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Conditional BuildStep Plugin • Ever wanted to have more control whether a step should be executed or not? • Want to reduce the number of jobs doing nearly the same thing? • Add a conditional buildstep which acts as a container. It will allow you to define a condition controlling the execution of the step(s).
  44. 44. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf
  45. 45. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Builds overhead • As you remember we have a deployment almost every day • Leads to enormous overhead How we can reduce the amount of time handling different version of our products ?
  46. 46. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Snapshot deployment • For rapidly moving code where bug fixes and enhancements are coming fast. • Reduce the amount of framework based checkout and builds. • Less framework variation to maintain. Always Moving forward ….
  47. 47. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf IDI Continues integration 'working copy'
  48. 48. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Lazy Developers How can I enforce my programmers to enter Jenkins before and after commit?
  49. 49. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Using the Jenkins Game plugin Jenkins email build notifications
  50. 50. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Lazy Developers What about code review?
  51. 51. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Code Inspection - Warnings Plugin
  52. 52. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Code Inspection - FindBugs plugin
  53. 53. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf If nothing helps Chuck Norris Plugin will!
  54. 54. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf 3 or 4 years ago… JAM – Program language using which solid part of CRM was written and still exists and even expands  – Final JAM product is EXE file JAM Deployment: – Every developer coping his EXE file to production directory. Problems: – Bugs in production – No tracing mechanism – No mechanism who checks correct integration between modules
  55. 55. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Jenkins is not just for java CI • Source Code Management: – All locally managed sources will be placed under new SVN Repository . • Triggers : – Check Poll SCM for code changes every minute. • Build: – Create buildAllModule.bat [$ENV] to handle JAM build in a generic way.
  56. 56. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf
  57. 57. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf And the same for the deployment phase
  58. 58. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Mobile CI Plans for the future
  59. 59. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Hybrid mobile development • Build the UI with jQuery Mobile CSS 3 and HTML5 • Connect to any REST-based API Optionally, use any PhoneGap API • Export the app as mobile Web, Android or iOS using PhoneGap
  60. 60. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Mobile CI Concerns - Building • Creating the IOS native artifact (PhoneGap) need to run under MAC. • In order to do so we need to create a Xcode project (using Xcode plugin). • Our Jenkins server runs over Linux VM and cannot perform Xcode project. SO…
  61. 61. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Distributed builds • Jenkins supports the "master/slave" mode, where the workload of building projects are delegated to multiple "slave" nodes. Jenkins Linux VM MAC computer Execute the Xcode job using PhoneGap Define new Phonegap Xcode mobile job
  62. 62. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf A jenkins Node creation • Jenkins offers the option to hook other Jenkins instances as slaves via JNLP. • The Mac slave will connect to our “Master” and build all Xcode jobs on it.
  63. 63. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Mobile testing… • Mobile web: – JavaScript integration tests that invoke RESTful services on the server using RestEasy. • Native applications: – Need to support different Testing ENVS.
  64. 64. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Mobile CI Concerns • Deployment phase: – No jenkins PhoneGap integration so far. – Need to support our own Markets / AppStore for development ENV’s.
  65. 65. Jenkins User Conference Herzelia, July 5 2012 #jenkinsconf Thank You To Our Sponsors Platinum Sponsors Silver Sponsor

×