02 - Build and Deployment Management


Published on

Published in: Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

02 - Build and Deployment Management

  2. 2. PURPOSE OF CURRENTTRAINING Why do we need to manage builds Theory and practice of build and deployment management Why do we need to manage deployments 2
  3. 3. CONTENTS Build management  What is build?  Why do we need to build?  Build types.  Tools and their specifics: Ant, Maven, make, etc  Tips Deployment management  Build vs deployment  Why do we need to deploy?  How to deploy and when?  Tools that might be useful 3
  5. 5. WHAT IS BUILD From Process To of conversion run at the end-user Source code Artifacts workstation 5
  6. 6. WHY DO WE NEED TO BUILD? Build RESULT OF Configuration  defined set of components having specific versions  components = functional units  components and their versions are chosen to meet specific objectives or tasks Main objectives 6 Functionality Performance
  7. 7. WHY DO WE NEED TO BUILD?Conclusion. Why we need to build is:  To incorporate additional functionality (feature)  To reach defined level of performance (as an example of non-functional requirement)And what about data?  Data has its own lifecycle  Which is incorporated into the database integration process 7
  8. 8. WHY DO WE NEED TO MANAGEBUILDS? When application grows in size, it becomes more complex As a result, it turns out that routine daily activities require additional management Organizing build process is one of such activities Therefore, build process also requires management What means complex application? From configuration management point of view, most complex application incorporate all possible variant 8 properties
  9. 9. BUILD VS VARIANTBuild: install.msiVariant: AppName-1.1.8.en.x32_beta.msiReal world example:HTC_Touch_Diamond2_WM6.5_RUU_Topaz_S_HTC_RUS_2.16.411.2_Radio_Sign_Topaz_61.44tc.25.32_4.49.25.17_Ship.exe 9
  10. 10. CONNECTION BETWEENCONFIGURATION AND VARIANTS Functional requirements for the variant being built #1 #2 #3 … #N version Maturity Platform variant OS Localization build Build type configuration + source code changes (changeset) 10 static runtime
  11. 11. WHAT IS VARIANT Variant is the result of the build characterized by the set of specific properties and their values 11
  12. 12. VARIANT PROPERTIES Version Revision Date Build type: debug or release Operating system, hardware platform (x32, x64, …) Localization (en, ru, ua, …) Platform (development, test, integration, production) Maturity (alpha, beta, release candidate, stable) Issue number (fix#387) License (MIT, GPL, BSD, CC, …) Project ID (EPM-SIGN) + Discipline ID (PD) 12
  13. 13. EXAMPLE.PLAIN DOCUMENT Build #2 Properties: • Language (en) • Date (24-12-2010) • Revision (4) Ctrl + N Ctrl + S Ctrl + S Ctrl + S Build #1 Properties: 13 • Language (en) • Date (22-12-2010) • Revision (3)
  14. 14. DEMO EXAMPLE 14
  15. 15. EMERGED QUESTIONS Why open source repositories are not structured properly? How to assign unique marker (version) for our changes? What repository model should we use? How do we need to manage experimental changes or development? What is initial codebase? 15
  17. 17. WHY DO WE NEED TO MANAGEBUILDS? Builds should be CRISPy! CRISP means:  Complete  Repeatable  Informative  Schedulable 17  Portable
  18. 18. HOW TO MANAGE BUILDPROCESS?Define what build tasks you needDefine what build types you needChoose build phases you will needChoose build toolStructure your build file 18
  19. 19. HOW TO MANAGE BUILDPROCESS?Define what build tasks you needDefine what build types you needChoose build phases you will needChoose build toolStructure your build file 19
  20. 20. BUILD TASKS BY PRIORITY Compilation Deployment  DB integration  Unit testing  Code coverage  Static analysis  Source code metrics  Dynamic analysis  Documentation generation  … 20
  22. 22. BUILD VS DEPLOYMENT Deployment is the process of software installation (software system available to use) While build is the process of installation artifacts creation Is this always true? Not really For interpreted languages (no compilation phase) there might be no significant difference between build and deployment But it is more convenient to think of all related activities (unit-testing, code coverage, inspections, etc) as a build, not deployment. 22
  23. 23. RESULTS OF THE BUILD Executables  .exe, .bin, .msi, .cab, … Libraries  .dll, .lib, .so, … Archives  .zip, .tar.gz, .bz2, .7z, .ear, .jar, .war, .sar, … Packages  .rpm, .deb, … 23
  24. 24. RESULTS OF DEPLOYMENT  Installed application  Updated database  Added incremental changes  Unpackaged library  Loading and installing dependencies … 24
  25. 25. BUILD VS DEPLOYMENTBuild Deployment Runs on single machine  Might be executed on several machines Does not require remote  Requires connection to connection the remote server Creates application  Installs application installation Makes sense always  Makes sense only in case of distributed or remote application 25 Primary  Secondary
  26. 26. HOW TO MANAGE BUILDPROCESS?Define what build tasks you needDefine what build types you needChoose build phases you will needChoose build toolStructure your build file 28
  27. 27. BUILD TYPESCLASSIFICATION BY: Use •Private (developers build) •Integration (centralized build) •Release (delivery to the end-user) 29
  28. 28. BUILD TYPESCLASSIFICATION BY: Structure, stage, result • Pre-build (ensuring zero build errors: checking space, clean, …) • Post-build (build verification tests, security tests, …) • Clean (rebuilding whole project from scratch) • Incremental (only changed files and artifacts) • Broken 30
  29. 29. BUILD TYPESCLASSIFICATION BY: Staging • Lightweight (recompilation, unit- tests, …) • Heavyweight (deployment, system tests, inspections, …) 31
  30. 30. HOW TO MANAGE BUILDPROCESS?Define what build tasks you needDefine what build types you needChoose build phases you will needChoose build toolStructure your build file 32
  31. 31. BUILD PHASES  Pre-build  Clean  Init  Check disk space Build  Compilation  Deployment  Post-build  Inspections  Documentation generation  Checking build errors  Security tests  Tagging 33
  32. 32. HOW TO MANAGE BUILDPROCESS?Define what build tasks you needDefine what build types you needChoose build phases you will needChoose build toolStructure your build file 34
  33. 33. BUILD TOOLSGNU Autotools Autotools alternatives• make • pymake• automake • CMake• autoconf • Cons• autoheader • SCons• libtool • qmake• gettext • makepp• gcc • JAM• pkg-config • waf 35
  34. 34. BUILD TOOLSJava based• Ant• Maven• Gant.NET based• MSBuild• Nant• Byldan• NMavenOther• Phing (PHP) 36• Rake (Ruby)
  35. 35. GNU AUTOTOOLS Did you build applications from sources on UNIX platform? What steps did you need to accomplish this? 1. ./configure 2. make 3. make install When does actual compilation happens? Right. During make command execution make is the father of all build tools Heart of the build with make is the makefile 38
  36. 36. GNU AUTOTOOLSMAKEFILE EXAMPLEtarget1 target2 target3 : prerequisite1 prerequisite2 command1 command2 command3myprogram: main.o part1.o part2.o gcc -o myprogram main.o part1.o part2.opart1.o: part1.c part1.h header.h gcc -O -c part1.cpart2.o: part2.c header.h gcc -O -c part2.cmain.o: main.c header.h gcc -O -c main.c 39clean: rm -f myprogram main.o part1.o part2.o
  37. 37. GNU AUTOTOOLS LIST autoconf  creates a configuration script for a package from a template file  From Makefile.in to Makefile automake  Creates template file from another template file  From Makefile.am to Makefile.in libtool  helps manage the creation of static and dynamic libraries gettext internationalization and localization library gcc (GNU compiler collection) 40  default compiler used for building from sources
  38. 38. APACHE ANT Implemented in Java For the purpose of building java applications And for the purpose of make replacement Advantages over make: Cross-platform, does not require buildfile generation Resolves circular dependencies Works with file hierarchies Easily determines outdated files Conforms to “java way of thinking” 41
  40. 40. APACHE MAVEN Principles:  Convention over configuration  Declarative execution  Reuse of build logic  Coherent organization of dependencies Features:  Project Object Model  Extensive plugin architecture  Predefined build lifecycle 43  Dependencies management
  42. 42. MAVEN VS ANT Maven Ant Description of project  Development of a build (convention over configuration) script per project Invocation of defined goals  Invocation of project (targets) specific targets Project knowledge  "Just" the build process build lifecycle, standard  too complex scripts project layout reusable  scripts are not reusable plugins, repositories 45
  43. 43. HOW TO MANAGE BUILDPROCESS?Define what build tasks you needDefine what build types you needChoose build phases you will needChoose build toolStructure your build file 46
  44. 44. BUILD FILE STRUCTURE Tasks Targets  compileCompilation  deployDeployment  deploy-db DB integration  run-tests Unit testing  check-coverage Code coverage  run-inspections Static analysis  gather-metrics Source code metrics  performance-tests Dynamic analysis  generatedoc 47 Documentation generation …  …
  46. 46. BUILD TYPES AND BUILDFILESTRUCTUREUse types: private, integration, release 49
  47. 47. LOCAL BUILD VS PRIVATEBUILD? Local build: What is the difference?  Compilation Omitting unnecessary steps:  Unit-testing Compilation Deployment Private build: Database integration  Compilation Loading initial data  Deployment Unit-testing  Database integration Simple static analysis  Loading initial data  Unit-testing 50  Simple static analysis
  48. 48. LOCAL BUILDLocal build is required after minor source code changes forthe purpose of having executable artefact Compilation Deployment Database integration Loading initial data Unit-testing Simple static analysis 51ant compile run-tests
  49. 49. PRIVATE BUILDPrivate build assumes that changes should be deployed tothe application container or web-server Compilation Deployment Database integration Loading initial data Unit-testing Simple static analysisant compile deploy deploy-db load- 52 data run-tests inspections
  50. 50. PRIVATE BUILD 53
  51. 51. INTEGRATION BUILDIntegration build assumes presence of continuousintegration practice as a key part of software lifecycle Compilation Deployment Database integration Loading initial data Unit-testing Static analysis Dynamic analysis 54 Documentation generation Gathering metrics
  53. 53. RELEASE BUILDRelease build is the result of development will be visible tothe end user Compilation Deployment Database integration Loading initial data Security tests integrity checks 56 performance tests
  54. 54. RELEASE BUILD 57
  56. 56. DEPLOYMENT TYPES Static  when the container starts Hot  Ability to deploy/undeploy deployables into a running container Incremental  Deploying only changed files/artifacts Remote  Requires connection via specified protocol Local 59  Does not require connection
  57. 57. DEPLOYMENT PROTOCOLS FTP/SFTP SSH/SCP No protocol Rsync (copy/move) Mail/e-mail (!) 60
  58. 58. DEPLOYMENT PHASESAND TASKS Deployment  Deploy  Restart application server Database integration  Generate ORM models, mapping  Dump data  Instantiate DB (DDL usage)  Data load into DB (DML usage, CRUD operations)  Migrate data, structure or both 61
  59. 59. INCREMENTAL DEPLOYMENT Rsync  synchronizes files and directories from one location to another SVN export only changed files  svn diff –summarize /path1 /path2 Deploy only files modified/added since last revision  svn status In most cases it’s just a… HEADACHE! 62
  60. 60. SIMPLE DEPLOYMENTWORKFLOW WC (working copy, trunk) local deployment instance db 63
  61. 61. LOCAL DEPLOYMENTWORKFLOW WC (working copy, trunk) local deployment dev instance test instance prod instance 64 dev db test db prod db
  62. 62. REMOTE DEPLOYMENTWORKFLOW + + + dev test prod local deployment WC (working copy, trunk)remote deployment + + + 65 dev test prod
  63. 63. MANAGE YOUR DEPLOYMENTS1. Define what deployment type you will use (inherits from corresponding build type)2. Define what deployment tasks you need3. For each deployment type prioritize chosen tasks4. Define destinations (production, test, development)5. Map destinations and deployment types6. Configure deployment destinations environment 66
  65. 65. DEVELOPMENT WORKFLOW EXAMPLE Deployed application WC (working copy) Repository svn up r1049 ant deploy-private r1050 r1051 r1052 development changeset (#146)Second deployed … manual merge application svn up r1126 ant deploy-private –Dinstance=2 resolve conflicts svn commit OK? yes 68 no make corrections manual merge svn commit
  66. 66. DEVELOPMENT WORKFLOWEXAMPLE NOTES Does this example remind you something? Yes! Distributed version control What is the point? Svn working copy is the repository While deployed application is a working copy Difference is that all operations between „repository‟ and „working copy‟ are manual And operations history is not tracked 69
  67. 67. DEVELOPMENT WORKFLOWEXAMPLE NOTES Why do we need two builds? To ensure there are no integration errors Because of “Don‟t break the build“ ruleAdvantages: You see your source code many times during merge You can find errors/inconsistencies easily Keeping unfinished development both safe and actualFlaws: A lot of time spent before commit Works only for web-projects written in interpreted languages 70
  68. 68. CONCLUSION71
  69. 69. BEST PRACTICES Separate build from IDE Run faster tests first  Introduce staging into build process Fail build fast  Organize build targets by priority Build components separately  Recursive build Centralization of project assets  Put sources to sources (VCS)  Put binaries to binaries (dedicated repositories) Consistent directory structure 72  Develop project structure conventions
  70. 70. BUILDS AND DEPLOYMENT MANAGEMENT1. Define what build/deployment tasks you need2. Define what build types you need3. Choose build/deployment phases you will need4. Prioritize chosen tasks with regard to corresponding build types5. Choose build tool6. Structure your build file7. Define deployment destinations8. Map build types and deployment destinations9. Configure deployment destinations environment 73
  71. 71. RECOMMENDED READING1. The Build Master: Microsofts Software Configuration Management Best Practices By Vincent Maraia 74
  72. 72. RECOMMENDED READING2. Ant: The Definitive Guide, 2nd Edition by Steve Holzne 75
  73. 73. RECOMMENDED READING3. Maven: The Definitive Guide by Timothy M. OBrien 76
  74. 74. USEFUL LINKS1. http://freshmeat.net/articles/build-and-release-management - Build and release management2. http://onjava.com/pub/a/onjava/2003/12/17/ant_bestpractices.ht ml - Ant best practices3. http://www.sonatype.com/books/mvnref-book/reference/public- book.html - Maven complete reference4. http://repo.exist.com/dist/maestro/1.7.0/BetterBuildsWithMaven. pdf - Great book about Maven5. http://kent.spillner.org/blog/work/2009/11/14/java-build- tools.html - Ant vs Maven6. http://martinfowler.com/articles/rake.html - article about rake by Martin Fowler7. http://www.finalbuilder.com/finalbuilder.aspx - FinalBuilder home page 77