Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

03 - Continuous Integration

2,725 views

Published on

Published in: Technology
  • Be the first to comment

03 - Continuous Integration

  1. 1. CONTINUOUS INTEGRATION
  2. 2. TRAINING GOALS Show that there is no way present-day project could be successful without usage ofcontinuous integration practice Establish connection between CMMI product integration process area and continuous integration practice 2
  3. 3. TRAINING PLAN1. What is continuous integration?2. Why do we need continuous integration?3. Prerequisites for continuous integration process4. General CI workflow5. How does continuous integration affect our development process?6. Tools and their features7. When CI is not effective?8. We have "true CI". What next?9. CI and CMMI product integration process area 3
  4. 4. INTRODUCTION TO CONTINUOUS INTEGRATION4 Basic ideas
  5. 5. WHAT IS CONTINUOUSINTEGRATION? 5
  6. 6. WHAT IS CONTINUOUSINTEGRATION?• Integration is when you make everything work together• Continuous is when you make everything work together very often 6• You would go mad if you had to do it manually• That‟s why it is not about manual actions, it is about automation
  7. 7. WHAT IS CONTINUOUSINTEGRATION? Software engineering practice Another step to the Approach development helping to process reduce risks automation 7
  8. 8. WHY DO WE NEED CONTINUOUSINTEGRATION? Martin Fowler:… team integrate their work frequently,… leading to multiple integrations per day.… this approach leads to significantly reduced integrationproblems and allows a team to develop cohesive software more rapidly. CI is the one of the XP practices Allows to have fresh latest build 8
  9. 9. PREREQUISITES FOR CONTINUOUSINTEGRATIONNot everything you can continuously integrate 9
  10. 10. PREREQUISITES FOR CONTINUOUSINTEGRATION Single source code repo (VCS system) Build CI tool automation Separate Self-testing server app (unit- 10 (usually) tests)
  11. 11. GENERAL CI WORKFLOW BuildCommitting (compilation, unit- latest testing, db changes integration, etc) Update by Application is scheduler (on ready CI server) 11
  12. 12. GENERAL CI WORKFLOW changes detected build & update deploycommit ~ repository deployment server continuous integration server 12
  13. 13. GENERAL CI WORKFLOW Check modification Build application 13
  14. 14. TOOLS14 Instruments for continuous integration and their features
  15. 15. TOOLS CLASSIFICATIONCI toolsBuild management toolsUnit testing toolsInspection tools• Code coverage analysis• Static analysis (syntax check, code dependencies)• Copy/paste detectors 15Documentation generation tools
  16. 16. CI TOOLS  CruiseControl  CruiseControl.NET  CruiseControl.rb  JetBrains TeamCity  Apache Hudson  Apache Continuum  Atlassian Bamboo  FinalBuilder  phpUnderControl 16  XInc
  17. 17. CI TOOLS  CruiseControl  CruiseControl.NET  CruiseControl.rb  JetBrains TeamCity  Apache HudsonMost common, flexible, configurable, easy to  Apache Continuum use, de facto choice for Java projects  Atlassian Bamboo  FinalBuilder  phpUnderControl 17  XInc
  18. 18. CI TOOLS  CruiseControl  CruiseControl.NET  CruiseControl.rb  JetBrains TeamCity  Apache HudsonMost common, flexible, configurable, easy to  Apache Continuum use, de facto choice for .NET projects  Atlassian Bamboo  FinalBuilder  phpUnderControl 18  XInc
  19. 19. CI TOOLS  CruiseControl  CruiseControl.NET  CruiseControl.rb  JetBrains TeamCity  Apache HudsonMost successful adaptation of CI principle for the  Apache Continuum interpreted language (Ruby)  Atlassian Bamboo  FinalBuilder  phpUnderControl 19  XInc
  20. 20. CI TOOLS  CruiseControl  CruiseControl.NET  CruiseControl.rb  JetBrains TeamCity  Apache HudsonOne of the best CI tools:  Apache Continuum• pre-tested commit• build agents  Atlassian Bamboo• multi-platform builds  FinalBuilder• build dependencies  phpUnderControl• comprehensive statistics and reporting 20  XInc
  21. 21. CI TOOLS  CruiseControl  CruiseControl.NET  CruiseControl.rbTagging after successful build, distributed builds  JetBrains TeamCity  Apache Hudson  Apache Continuum  Atlassian Bamboo  FinalBuilder  phpUnderControl 21  XInc
  22. 22. CI TOOLS  CruiseControl  CruiseControl.NET  CruiseControl.rb Distributed builds  JetBrains TeamCity  Apache Hudson  Apache Continuum  Atlassian Bamboo  FinalBuilder  phpUnderControl 22  XInc
  23. 23. CI TOOLS  CruiseControl  CruiseControl.NET  CruiseControl.rbFocused on processJetBrains TeamCity just a feature  of building, CI is  Apache Hudson  Apache Continuum  Atlassian Bamboo  FinalBuilder  phpUnderControl 23  XInc
  24. 24. CI TOOLS  CruiseControl  CruiseControl.NET  CruiseControl.rb CI tools written in PHP for PHP projects  JetBrains TeamCity  Apache Hudson  Apache Continuum  Atlassian Bamboo  FinalBuilder  phpUnderControl 24  XInc
  25. 25. CI TOOLS  CruiseControl  CruiseControl.NET  CruiseControl.rbYet another CI tool written in TeamCity PHP projects  JetBrains PHP for  Apache Hudson  Apache Continuum  Atlassian Bamboo  FinalBuilder  phpUnderControl 25  XInc
  26. 26. CI TOOLS Too many tools All are great All have the same principle But each has specific featuresDiagrams, metrics, reporting Integration with SCM tools Notification Interface, usability Distributed builds Different ALM approaches 26
  27. 27. CI TOOLS. ALM APPROACH Requirements Initiation development & Design management Testing Integration Implementation Maintenance & Deployment Utilization 27 support
  28. 28. CI TOOLS. ALM APPROACH Requirements Initiation development & Design management Basic continuous integration Testing Integration Implementation Maintenance & Deployment Utilization 28 support
  29. 29. CI TOOLS. ALM APPROACH Requirements Initiation development & Design management Basic continuous integration Testing Integration Implementation Extended build management Maintenance & Deployment Utilization 29 support
  30. 30. BUILD TOOLS  Ant  NAnt  MSBuild  Maven  Make  Phing  Rake  … 30
  31. 31. BUILD TOOLS  Ant  NAnt  MSBuild  Maven Most common, flexible, configurable, easy to  Make use, de facto choice for Java projects  Phing  Rake  … 31
  32. 32. BUILD TOOLS  Ant  NAnt  MSBuild  Maven Most common, flexible, configurable, easy to  Make use, de facto choice for .NET projects  Phing  Rake  … 32
  33. 33. BUILD TOOLS  Ant  NAnt  MSBuild  Maven  Makede facto case for Visual Studio and TFS users  Phing  Rake  … 33
  34. 34. BUILD TOOLS  AntBuild management + dependencies management  NAnt for Java projects. Introduced POM concept.  MSBuild  Maven  Make  Phing  Rake  … 34
  35. 35. BUILD TOOLS  Ant Standard de facto for UNIX applications  NAnt  MSBuild  Maven  Make  Phing  Rake  … 35
  36. 36. BUILD TOOLS  AntImplementation of build management tool in PHP  NAnt for PHP projects  MSBuild  Maven  Make  Phing  Rake  … 36
  37. 37. BUILD TOOLS  AntImplementation of build management tool in Ruby  NAnt for Ruby projects  MSBuild  Maven  Make  Phing  Rake  … 37
  38. 38. UNIT TESTING TOOLS  Junit  NUnit  CppUnit  PHPUnit  SimpleTest  JSUnit  J3Unit 38
  39. 39. INSPECTIONS TOOLS.TEST COVERAGE Cobertura Atlassian Clover jTest JCoverage CodeCover EMMA Parasoft Insure++ Ncover Xdebug 39 Coverage.py
  40. 40. INSPECTIONS TOOLS.TEST COVERAGE Cobertura Atlassian Clover jTest JCoverage Java CodeCover EMMA Parasoft Insure++ C++ Ncover C# Xdebug PHP 40 Coverage.py Python
  41. 41. INSPECTIONS TOOLS.STATIC ANALYSIS PMD FindBugs JLint FxCop Checkstyle ReSharper PHP_CodeSniffer Yasca 41 NDepend
  42. 42. INSPECTIONS TOOLS.COPY/PASTE DETECTORSCPD (Copy paste detector)CheckstyleSimianJplagAtomiqClone diggerPBA 42
  43. 43. DOCUMENTATION GENERATIONTOOLS Doxygen JavaDoc NDoc CppDoc phpDocumentor pyDoc RDoc 43
  44. 44. OTHER TOOLS Source code Dead code metrics (loc) detectors Profiling … 44
  45. 45. CI IN EPAM MICROSOFTTECHNOLOGIES DIVISIONProject Project Product Primary CCNET Code Analysis Unit Test Copy & Automat Automate Language tools testing coverage Paste e build deploymen tools detectors delivery tACT-INT C# x - x (Unit) - - - -ACT-LVI C# x x (FxCop, PBA) x - - - -CON-GEN C#, ASP.NET - - - - - - -EPM-UTIL C#, ASP.NET x x (FxCop,PBA) x (NUnit) x (NCover) x (Simian) + +ESS-MSFT - - - - - - -FSTIDX C#, ASP.NET x x (PBA) x - - - -ICPDP C# x - - - - - -LOGIDEX Java, J#, C#, ASP.NET - - - - - - -LSEEC C# x x (FxCop, PBA) x (NUnit) - x (Simian) - +MIS-OPC C# x x x - - - -MIS-SPR C# x x x (Nunit) x - - -MultAPI VBScript - - - - - - -MultCons C# x x (FxCop, PBA) - - x (Simian) - -MultData C#, C++, HTML, VB x - x - - - -MultEDG C++, C# - x (FxCop) x - - x -Mult-IPAS C#, C++ x x (FxCop) - - - - -MultRIN C# - - x - - - xMult-Torn C++ - - x - - - -
  46. 46. CI IN EPAM MICROSOFTTECHNOLOGIES DIVISIONProject Project Product Primary CCNET Code Analysis Unit Test Copy & Automat Automate Language tools testing coverage Paste e build deploymen tools detectors delivery tRTRS-AAAM C# - - - - - - -RTRS-MTST C#, ASP.NET - x (FxCop) - - - - -RTRS-RKSD C# x x - - x (Simian) - -RTRS-RKWM C# - x - - - - -TAK-PBLD C# - x (FxCop) x (Nunit) - - - -TRR-ODC (O2I) C#, ASP.NET x x (FxCop,PBA) x (Nunit) - x (Simian) - -TRR-ODC (SapSn) C# x x (FxCop,PBA) - - x (Simian) - - 46
  47. 47. MOST TYPICAL QUESTIONS All the above-described info sounds cool. I feel that it might be useful in my project. But I am not sure I can find enough resources to start and support this activity. EPM-BET project is started long time ago for this purpose. It provides corresponding resources and support There are too many different tools. They are all different. How could I use them both independently and all at once? I recommend to use AgileSCM approach. It uses 47 unified versions numbering which could be used by any mentioned tool.
  48. 48. ADVANCED CONTINUOUS INTEGRATION48 More complex issues related to the continuous integration
  49. 49. HOW DOES CI AFFECTDEVELOPMENT PROCESS? Commit policy (mainlines are on the watch) Dont break the build rule (make local build) Everyone commits to the mainline everyday Keeping the build fast  Run fast tests first We can automate deployment  Build is primary, deployment is secondary Continuous design 49
  50. 50. “DON’T BREAK THE BUILD”RULE Test code properly before checking in Avoid depending on a local resource that is not under version control or does not exist on the target computer. Write tests to cover common problems Make sure local inspections were run successfully Broken build in most cases means that it is impossible or not recommended to use corresponding codebase until it has been fixed 50
  51. 51. “DON’T BREAK THE BUILD”OBSESSION It is often impossible to reproduce integration problem without checking in. You might spend A LOT of time looking for a failure reason in someone‟s else module During that time you will be the one WHO FAILED THE BUILD If you want to avoid such situations, every checking in becomes a nightmare Attitude to the “don‟t break the build” rule may be a detector of how mature the team is. 51
  52. 52. “DON’T BREAK THE BUILD”OBSESSION. SOLUTION Nightly builds instead of builds „on commit‟ Team needs more strict and less strict variations of the “don‟t break the build” rule during different SDLC phases There are architectural issues to pay attention to Sometimes several small projects should be started instead of one large 52 Build policy is to be developed
  53. 53. WHEN CI IS NOT EFFECTIVE? DVCS Experimental development Small projects Interpreted languages without unit-tests When database is being changed too often Types of projects  Documentation (BA, etc)  DB  Support & maintenance 53
  54. 54. WE HAVE "TRUE CI".WHATS NEXT? Building tags Separate builds having different maturity  levels  Codebase structure filtering  Triggering build only for changes in specified list of folders Integration with issue tracking systems Different approach for different SDLC 54 phases
  55. 55. WE HAVE "TRUE CI". WHAT’S NEXT? 1.x.x 2.x.x/trunk PA 1.x.0 1.x.3 2.x.0 A 1.x.1 1.x.4 2.x.1 builds B 1.x.2 1.x.5 2.x.2 /1.x.x AR 1.0.0 BR 1.0.1 RC 1.0.2 1.0.3 releases ST 1.0.4 56 /1.0.x
  56. 56. WE HAVE "TRUE CI". WHAT’S NEXT? 1.x.x 2.x.x/trunk 1.x.0.0 1.x.0.1 1.x.1.0 1.x.1.1 1.x.2.0 2.x.x.0 2.x.x.1 2.x.x.2 2.x.x.3 2.x.x.4 … 2.x.0.0 2.x.0.1 2.x.1.0 2.x.1.1 integration (dev) 2.x.0 2.x.1 2.x.2 builds CI server 1.x.0 1.x.1 1.x.2 1.x.3 1.x.4 1.x.5 builds 1.x.2.1 … 1.x.3.0 1.x.3.1 1.x.4.0 1.x.4.1 1.x.4.2 1.x.4.3 integration (dev) /1.x.x CI server releases 1.0.0 1.0.1 1.0.2 1.0.3 1.0.4 integration (rel) 1.0.x.0 1.0.0.0 1.0.0.1 1.0.1.0 1.0.1.1 1.0.2.0 1.0.2.1 1.0.3.0 1.0.3.1 57 /1.0.x
  57. 57. WHAT NEXT? VERSIONSNUMBERING PATTERNS N.x.x.L pilot integrationintegration N.x.K.L development integration N.?.?.L N.M.x.L pre-release integration N.M.K.L release integration 2.0.x.0 2.0.x.1 2.0.0 2.x.x.0 2.x.x.1 2.x.x.2 2.x.x.3 2.x.x.4 2.x.0 … … 1.x.1.0 1.x.1.1 1.x.2 1.1.x.0 1.1.0 1.x.2.0 1.x.2.1 1.x.3 1.x.3.0 1.x.3.1 1.x.4 dev rel 58 N.x N.M 1.0.x.0 1.0.0 1.0.0.0 1.0.0.1 1.0.1
  58. 58. 59
  59. 59. CONCLUSION Thereis a lot of possibilities to make your continuous integration better:  Several integration streamlines (not only trunk)  Development and release integration streamlines  Introduce version number consisting of four symbols  Proper version number inheritance  Integration of builds and releases  …
  60. 60. CONCLUSION CI is another SCM tool (practice) Middle size project should definitely have it. It is used mostly in agile world Agile SCM is the result of attempt to understand what version should have artifacts which are being built by CI server
  61. 61. AFTERWORD62
  62. 62. RECOMMENDED READING1. Continuous Integration: Improving Software Quality and Reducing Risk by Paul M. Duvall, Steve Matyas, Andrew Glover 63
  63. 63. RECOMMENDED READING2. Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation by By Jez Humble, David Farley 64
  64. 64. RECOMMENDED READING3. Release It!: Design and Deploy Production-Ready Software by Michael T. Nygard 65
  65. 65. RECOMMENDED READING4. Pragmatic Project Automation: How to Build, Deploy, and Monitor Java Applications by Mark Clark 66
  66. 66. USEFUL LINKS http://martinfowler.com/articles/continuousIntegration.ht ml - the main article about CI from Martin Fowler http://www.infoq.com/news/2009/03/Continuous- Deployment - Beyond Continuous Integration: Continuous Deployment http://radar.oreilly.com/2009/03/continuous-deployment- 5-eas.html - Continuous Deployment in 5 easy steps http://www.proudlyserving.com/archives/2007/10/fear_of _a_broke.html - „fear of the broken build‟ effect http://lib.custis.ru/Continuous_Integration - wiki about Continuous Integration in Russian http://habrahabr.ru/blogs/php/91777/ - Providing quality for web applications (rus) http://www.youbrokethebuild.com/ - great posters created with the purpose of motivation people avoid breaking the build 67

×