Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Anatomy of a Build Pipeline


Published on

You've heard about Continuous Integration and Continuous Deilvery but how do you get code from your machine to production in a rapid, repeatable manner? Let a build pipeline do the work for you! Sam Brown will walk through the how, the when and the why of the various aspects of a Contiuous Delivery build pipeline and how you can get started tomorrow implementing changes to realize build automation. This talk will start with an example pipeline and go into depth with each section detailing the pros and cons of different steps and why you should include them in your build process.

Published in: Technology

Anatomy of a Build Pipeline

  1. 1. Sam November 7, 2012
  2. 2. Thanks to Mike McGarr andExcella Consulting for hosting!!
  3. 3. Sam Brown 11+ Years as a Java developer with commercial and federal clients Practicing continuous integration/continuous delivery for ~6 years DevOps Evangelist at Excella ( Certified Scrum Master Puppet Certified Professional
  4. 4. Basic components of an automated enterprise Continuous integration Dependency management Automated build toolsto build... Shared API libraries Custom web applications Products
  5. 5. “The purpose of a pipeline is to transport some resourcefrom point A to point B quickly and effectively with minimalupkeep or attention required once built” – me So how did „pipelines‟ get applied to software? Let‟s try a few changes to this statement...“The purpose of a pipeline is to transport from to quickly andeffectively with minimal upkeep or attention required oncebuilt” – me
  6. 6. Build pipelines require measurements and verification ofthe code to ensure: Adherence to standards Quality proven through testing A product that meets user‟s needsThe purpose is not just transport, but to ensure that ourproduct is high-quality, prepared for the environment it willreach, and satisfies the end-user.
  7. 7. “An automated manifestation of the process required to get yourteam’s application code to the end-user, typically implemented viacontinuous integration server, with emphasis on eliminatingdefects” – me (again)
  8. 8.  …in fact, NONE ARE! Build pipelines will vary as much as applications Different teams have different needs Simplicity is key One Size Fits All!
  9. 9. Repeatable, automated, process to ensure that application code is tested, analyzed, and packaged for deployment.
  10. 10.  System of record Just do it! Take advantage of commit hooks Build from trunk and reduce server-side branches Tag often Don‟t check in broken code!
  11. 11. Purpose: Integrate, build and unit test code for quickfeedback Best Practices  Runs in under 10 minutes (rapid feedback)  Unit tests do not require external resources  Run on EVERY developer check-in  Fixing broken builds is the top priority!  Gamification to drive adoption  80% test coverage or BETTER Challenges  LOTS of builds  False sense of security  Writing tests is hard
  12. 12. Purpose: Test component and/or external resourceintegration Best Practices  Test connectivity with external resources  Test frameworks load correctly  Test application components work together  Test configuration  Fewer integration tests than unit tests Challenges  External resources may not be available in all environments ○ Mock locally  Can be time consuming ○ Use local resources ○ Separate short/long running tests
  13. 13. Purpose: Use automated tools to inspect code Best Practices  Check syntax  Find security vulnerabilities  Record test coverage  Discover complexity  Optional: Fail based on a metric  Optional: View technical debt Challenges  Not all code analysis tools are free  Learning/installing new tools
  14. 14. Purpose: Label code and package asdeployableBest Practices  Labeling allows you to go back in time  Packaging code for deployment  Reduce complexity by combining steps  NO configuration in package -> Package once, deploy multiple Challenges  Labeling can be resource intensive  Many packaging options
  15. 15. Purpose: Make artifacts available fordeployment or available to other teams Best Practices  Publish a versioned artifact  Make repository available  Reduce complexity by combining steps Challenges  Requires initial complex setup  Security requirements around exposing artifacts ○ Use a tool with security built-in like Nexus
  16. 16. Repeatable, automated, process to ensure that our target environment is properly constructed for our application(s).
  17. 17. Purpose: Check syntax and compile prior toapplication Puppet Lint – Static format checker for Puppet manifests No-op Test Run – Ensure that manifest compiles Challenges  Puppet-lint requires a ruby-based environment  No-op test needs production-like VM  Long feedback loop
  18. 18. Purpose: Test infrastructure in a prod-like environment Puppet Apply –Puppet application against VM that mimics DEV/TEST/PROD Infrastructure Tests – Test your environment! Example tests:  Users and groups created  Packages installed  Services running  Firewall configured Challenges  Long feedback loop  Yet another language (cucumber/rspec/other)  VM must be up to date with DEV/TEST/PROD
  19. 19. cucumber-puppet rspec-puppetFeature: Services require spec_helperScenario Outline: Service should be running and bind to describe logrotate::rule doport let(:title) { nginx } When I run `lsof -i :<port>` Then the output should match /<service>.*<user>/ it { should include_class(logrotate::rule) } Examples: it do | service | user | port | should contain_file(/etc/logrotate.d/nginx).with({ | master | root | 25 | ensure => present, | apache2 | www-data | 80 | owner => root, | dovecot | root | 110 | group => root, | mysqld | mysql | 3306 | mode => 0444, }) end end mber-puppet/wiki
  20. 20. Repeatable, automated, process to ensure that our application isproperly installed in the target environment and that the application meets acceptance criteria.
  21. 21. Purpose: Test acceptance criteria in a prod-likeenvironment Puppet Apply – Apply Puppet manifests including deploying application Run Acceptance Tests – “End-to-end” testing  End-user perspective  Meets user-defined acceptance criteria  Possible tools: Cucumber, Selenium, Geb, Sikuli Challenges  Maintain a production-like VM  Acceptance tests brittle ○ Test at the right level  Acceptance tests long running ○ Run nightly
  22. 22. Purpose: Label application and infrastructurecode, deploy to DEV environment Label Release Candidate – Known “accepted” versions will be deployed together Deploy to DEV – Automated deployment  Infrastructure AND application Challenges  DEV updating, not deployed from scratch ○ Create tests for ALL possible scenarios  Security ○ Work with security early and often!
  23. 23. Simplified process to support streamlined deployments to TEST and PRODUCTION
  24. 24. Purpose: Enable the test team to pull thelatest code Pull-based deployment Manual Testing/Approval Challenges  Enabling test team is a paradigm shift  Producing changes too fast ○ Create good release notes ○ Not every build needs manual testing
  25. 25. Purpose: Enable operations team to pull thelatest code into production “Push-button” deployment to production  Requires testing approval Challenges  Audit/security check before deployment ○ Discuss with operations ○ Automate as much as possible and prudent  Paradigm shift for operations, TOO EASY! ○ Engage the operations team as early and often  Rollback/Roll forward strategy ○ Easier with RPM‟s, I prefer roll forward
  26. 26.  Remove human error Repeatability tests and improves the process Visibility from code to deployment Baked-in quality Metrics, metrics, metrics Rapid and constant feedback Releases are non-events
  27. 27. Why do we store old/obsolete versions? Rollback Auditing History? Any other reason?My view: Store only the latest build and current production release Bugs fixed in latest version (Almost) impossible to reproduce environments Version control has historyException: Other teams dependent on a previous version Store major/minor revisionsReasoning: In a continuous delivery environment, delivering frequentlyallows you to keep moving forward with new features AND bug fixes!
  28. 28.  Put EVERYTHING in version control Start simple, up your unit test coverage. Analyze your code in order to focus Install CI and start with two build steps Start and maintain a wiki And lastly…
  29. 29.  @SamuelBrownIV