Continuous Delivery

6,487 views
6,293 views

Published on

Getting software released to users can be risky, time-consuming and painful. The solution is the ability to deliver reliable software continuously through build, test and deployment automation, and through improved collaboration between developers, testers and operations. In this tutorial we will present principles and technical practices that enable teams to incrementally deliver software of high quality and value into production whenever they want, and extremely fast. The size of the project or the complexity of its code base does not matter.

In the first half of the tutorial we will introduce the concepts of continuous delivery, through continuous integration; and automation of the build, test and deployment process. We will also go through som basic principles and patterns for building automatable applications (architecture). We will cover experiences on team collaboration patterns and lastly; techniques for solving tasks such as an easy and comprehendible version control strategy.

The second half of the tutorial we will be working with automated provisioning of agile infrastructure, including the use of tools (puppet) to automate the management of testing and production environments. We will go through some scripting lessons examplifying how to implement zero-downtime deploys (… and rollback – if something goes wrong!), with examples in both bash and Ruby. Along with controlling the start, stop, restart lifecycles during deploys, we will also show some simple techniques for backups, logging, error handling, monitoring and verification of application health that can make the automation more robust.

We will also use servers "in the cloud" to demonstrate different techniques, and we hope to make it a fun day and to deliver software (examples) several times throughout the workshop.

Required knowledge: Agile/Lean basics, Linux basics, version control basics, maven basics.

Published in: Technology, Business
0 Comments
17 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,487
On SlideShare
0
From Embeds
0
Number of Embeds
60
Actions
Shares
0
Downloads
224
Comments
0
Likes
17
Embeds 0
No embeds

No notes for slide
  • 09:00 - 09:45: Intro, fortelleomprosjektenevåreogossselv, organisasjonnele ting, hvorfor CD?09:55 - 10:40: Intro tilutviklings-stack (VCS CI, nexus) oghvordanvelge en egnetapplikasjonsarkitektur (Jetty, Maven/Gradle/Rake)10:50 - 12:00: Health-check, monitoring, error handling, avoiding leakage between environments(Prod/test). Provisioning of infrastructure (Puppet).13:30 - 14:00: Controlled deployment, Zero-downtime-redeploy. Daytime deployment.14:10 - 15:00: Advanced topics: db without downtime. Wrap up.Tekniskoppsett:* Referanseappsomkjøreri Jetty med nginx for session drain.* Puppet image.* Nexus, Jenkins, Sonar
  • 1) Innledning- Vi stiller spørsmåltilsalen: Hvoroftedeployererdere? Hardereautomatisertdeployering? - Continuous Delivery vs. CI / CD(eployment), Agile manifest, etc- Automatisering - Viktighetavåvelgeautomatiserbarteknologi, etc- Like miljøer- Gjenproduserbarhet, osv
  • Opex: Typical
  • Hence: co-locate or get a proper communication structure between operations and development…
  • Cross-functionalCustomer on-siteSelf sufficientDev-opsSelf organizingTrust
  • Trust must be earned
  • 1) Innledning- Vi stiller spørsmåltilsalen: Hvoroftedeployererdere? Hardereautomatisertdeployering? - Continuous Delivery vs. CI / CD(eployment), Agile manifest, etc- Automatisering - Viktighetavåvelgeautomatiserbarteknologi, etc- Like miljøer- Gjenproduserbarhet, osv
  • Eksempelprosjekt:Behov: * En server somkankjøre to-fire ulikevirtuelleservereForslagtilmiljøstack:* Frontend: nginxeller apache* Applikasjon: se under* Database: mysqlellerpostgresql* Automatisert med puppetEnkel CRUD-webapp med litt basis-funksjonalitet* Integrasjon mot eksterntgrensesnitt (f.eks. Bring Fraktguide // postnummervalidering, etc).* Forslagtilenkel stack: Spring(opsjonellt), Spring JDBC, et webrammeverk (Jersey eller Spring MVC), * Deployesienkel jetty-war* Littrammeverkskode for konfigurasjon / init-scriptInteraktive features:* Feature toggle: Slåav / på JSON-grensesnitt* Feature branch: Utvidegrensesnitt med XML-supportEndring 1: Slåpåstylesheet, css, etc
  • * Usually don’t code
  • Push vs. pull –basert deploymentKonfigureringavmiljø: Jenkins? Nexus (pull)?
  • Push vs. pull –basert deploymentKonfigureringavmiljø: Jenkins? Nexus (pull)?
  • 3) Jobbing/Samarbeiditeamet(ne) ogkontinuerligintegrasjon- Innledning: versjonskontrollavprosjektet. Feature branching. Feature toggling. Littomteamsammensetninger: deployeselv, vs. ha en devoppåteamet, etc. Innledningtileksempelprosjektet.- Spm: Hvor mange har en drifter påteamet? Hvor mange brukergit? Hvor mange brukerclearcase? Hvor mange haransvar for ålevere? Hvor mange mener at man erferdignår man leverertil QA? Hvor mange harforretningsutvikling? Kanban? ITIL (UFF)?- Vi demonstrereroppsettav en applikasjoni et gittutviklingsmiljø:* Versjonskontroll for kontinuerlig levering* Byggserversomkjører tester ogdeployertil nexus* Applikasjonsoppsett med maven/rake/etc (applikasjonskonfigurasjon)* environment:setup (klargjøringav server for applikasjonen)- Muliginteraksjon: Vi girhvert team en enkeloppgave (f.eks. team 1 skalendre en kodelinjeog en test, team 2 skalslåpå en feature, team 3 skal merge inn en feature-branch i master/release, ...)
  • Code base is never production ready!When to release?WIP hinders us from putting ready features into productionWe are never done done!
  • What happens if you forget to merge the bug fix into development?What happens if features are delayed?
  • Grey arrows: Potentially shippable
  • Rember to merge back to develop!
  • Release branches frees up the development branch for further development.It also frees up the master branch for bugfixes.
  • - Innledning: Kortinnledningomhvorfordeterviktigåogsåautomatisereverifisering (isåstor grad somdetermulig)- Events vs. logging- Vi demonstrerernoenteknikker for åverifisere at applikasjoneneri god helse (GET http://node/health, GET http://node/version, logger feil =>...)- Enkeleksternovervåkningav http://app/health- Enkelnagios-monitoreringavmiljø- Eks: cucumber-nagios* Feilscenarie 1: Vi demonstrerer en applikasjonsfeilsomskjeri runtime (f.eks. DB-fuckup somkræsjerapplikasjonen) => (ERROR fraovervåkning) red - vi demonstereropprettelseavfiks, test for åhindreframtidigfeilog redeploy => green* Feilscenarie 2: Vi demonstrerer en brukerfeilsomskjeri runtime => red (ERROR iloggen) (f.eks. manglendevalideringoghvordandetteplukkesoppav log-monitorering) - vi demonsterermuligfiks, mer robust test for åhindreframtidigfeil, og redeploy => green
  • - Innledning: Vi fortellerom et par ulikemuligeløsninger, vi fortellerom de ulikestegene en applikasjonkanfeilei (bygg, test, release, upload, stop, start, verify (health), runtime). Littomsesjonerog draining avsesjoner (ta en node ned -> vent tilallesesjonererferdige -> fortsett redeploy).- Demonstrasjon: Vi deployerernyversjon (uten DB-endring) til de ulikemiljøene* environment:deploy (utrullingtilservere - push - med venting ogverifiseringavstegenei upload, stop, start, verify)* puppet / environment:update (utrullingtilservere - pull fra nexus - med manuellhåndteringav deploy tilnoderogevt. switchover strategi /f.eks. switche QA og PROD)- Muliginteraksjon: Teamene pusher sine endringerog vi demonstrererautomatisertkontinuerligutrullingav de ulikeversjonenesomblelageti 3)
  • - Innledning: vanskeligheter. It depends! Vanskeligereåautomatisere- Vi demonstrererautomatisering med liquibaseogf.eks. dbdeploy.- Vi demonstrerer expand/contract patternet for DB-migrering.- Vi demonstrerer read-only-state for systemet.
  • - Littomorganisasjonellekravogutfordringer, f.eks. Lean / Kanban / JIT / etc, ITIL, eksternedriftere, etc...- Littmeromverdien (fåutforretningsverdi, fåned lead time, fånedantallfeili prod, fiksefeil, prodsettepådagtid, sikkerhet
  • Jim HighsmithThoughtWorksco-author the Agile Manifestoco-founded the Agile Alliance, and the Agile Project Leadership Network.
  • Continuous Delivery

    1. 1. Continuous Delivery<br />Ole Christian Rynning & Stein Inge Morisbak<br />Roots 2011<br />
    2. 2. https://github.com/bekkopen/roots2011-continuous-delivery<br />
    3. 3. Agenda<br />Who are we, where do we come from and what do we work on?<br />What does Continuous Delivery really mean?<br />Collaboration, Version Control and Continuous Integration.<br />Controlled deployment<br />Zero-down-time redeploy (and roll-back).<br />Robust applications (architecture)<br />Provisioning of Infrastructure for Continuous Delivery.<br />Error handling, Monitoring and health verification.<br />Migrating databases.<br />Summary, organizational impacts and business value.<br />
    4. 4. Stein IngeMorisbak<br />Information Science, University of Bergen<br />10 years +<br />Operator (≈ 2)<br />Consultant (≈ 5)<br />NorgesGruppen Data (≈ 6)<br />Many roles:<br />Developer<br />Operator<br />Tech lead<br />Architect<br />Agile/Tech coach<br />Manager<br />
    5. 5. Digipost<br />National service for digital mail.<br />Replacement for physical box.<br />PostenNorge AS.<br />Launched april 4th.<br />Java.<br />13 developers on the team.<br />I’m tech lead.<br />Agile process.<br />Self organizing.<br />Continuous Delivery.<br />Deployment of new features every week.<br />
    6. 6. Ole Christian Rynning<br />Master IT, UiO<br />5 years +<br />Consultant (≈ 4)<br />Many roles:<br />Developer<br />UI Developer<br />Operator<br />Tech lead<br />Architect<br />Creative Leader<br />
    7. 7. Bring Development Team<br />Bring<br />Business side (B2B) of Norway Post<br />Consists of 8 specialists (Express, Parcels, Cargo, Frigo, Mail, …)<br />Several projects<br />Tracking (public, B2B, APIs, mobile applications) (6 rels 2011)<br />Mybring (reports, calculators, APIs) (2 rels 2011)<br />Fraktguide (B2B, B2C, price quoting service) (2 rels 2011)<br />Booking (Order products across all specialists) (42 rels 2011)<br />Java & Ruby<br />10 developers, 2 UX, 1/3 operators, 4 product managers, …<br />Post-Agile process<br />Delivers Continuously *<br />
    8. 8. What does Continuous Delivery really mean?<br />
    9. 9.
    10. 10. Principles behind the Agile Manifesto<br />We follow these principles:<br />Our highest priority is to satisfy the customer<br />through early and continuous delivery<br />of valuable software.<br />Welcome changing requirements, even late in <br />development. Agile processes harness change for <br />the customer's competitive advantage.<br />Deliver working software frequently, from a <br />couple of weeks to a couple of months, with a <br />preference to the shorter timescale.<br />Business people and developers must work <br />together daily throughout the project.<br />Build projects around motivated individuals. <br />Give them the environment and support they need, <br />and trust them to get the job done.<br />The most efficient and effective method of <br />conveying information to and within a development <br />team is face-to-face conversation.<br />Working software is the primary measure of progress.<br />Agile processes promote sustainable development. <br />The sponsors, developers, and users should be able <br />to maintain a constant pace indefinitely.<br />Continuous attention to technical excellence <br />and good design enhances agility.<br />Simplicity--the art of maximizing the amount <br />of work not done--is essential.<br />The best architectures, requirements, and designs <br />emerge from self-organizing teams.<br />At regular intervals, the team reflects on how <br />to become more effective, then tunes and adjusts <br />its behavior accordingly.<br />
    11. 11. Our highest priority is to satisfy the customer<br />through early and continuous delivery<br />of valuable software.<br />
    12. 12. What is Continuous Delivery?<br />Continuousdelivery is about putting thereleaseschedule in the hands ofthe business, not in the hands of IT. <br />Implementingcontinuousdeliverymeansmaking sure yoursoftware is alwaysproductionreadythroughoutitsentirelifecycle – thatanybuildcouldpotentially be released to users at the touch of a buttonusing a fullyautomatedprocess in a matter ofseconds or minutes.<br />- Jez Humble (http://continuousdelivery.com/)<br />
    13. 13. The Big Picture<br />
    14. 14. It’s all about Business Value<br />Reduce Operational Expenditure (OPEX)<br />Reduce manual bottlenecks <br />Reduce overhead in communication (bust dev-qa-ops silos)<br />Increased control over software’s life cycles<br />Reduce Capital Expenditure (CAPEX)<br />Increase/optimize utilization (i.e. virtualization)<br />Reduce startup costs<br />Increase Customer Satisfaction (Protection)<br />Improve usability, performance, security, requirements<br />Respond quicker to customer demands<br />Increase brand value (Revenue)<br />
    15. 15. Questions you need to answer<br />Are you able to put new featuresinto production within a reasonable amount of time (i. e. less than a day)?<br />You don´t have that requirement?<br />What about bug-fixes?<br />Would your customer be happier if she made a decision and saw it in production the same day?<br />Would your customer be happier if you could deploy without down-time?<br />Would you trust your deployment process more if you did it more often?<br />Would you feel more safe if you deployed fewer features to production at a time?<br />Would you feel more safe if you had less things that could go wrong?<br />Would you be happy with a heavy manual deployment process if you where to release several times a week?<br />Would operations be happier (and everyone else safer) if deployment was automated instead of documented.<br />Would you be happier (and not so lonely) if you could deploy during work hours instead of in the middle of the night?<br />Are you able to roll-back (alternatively roll forward) if deployment fails?<br />
    16. 16. Lean for Software Development<br />
    17. 17. The Challenge<br />How long would it take your organization to deploy a change that involves just one single line of code?<br />Do you do this on a repeatable reliable basis?<br />- Mary and Tom Poppendieck<br />
    18. 18. Conway’s law<br />...organizations which design systems [...] are constrained to produce designs which are copies of the communication structures of these organizations<br />- Melvin Conway, 1968<br />www.melconway.com/research/committees.html<br />
    19. 19. An Agile team<br />
    20. 20. Continuous Delivery team<br />Trust and collaboration.<br />Transparent (big visible displays, demos and retrospects).<br />Releases driven by business needs and fast feedback.<br />Fearless (config. mgmt., automated testing at all levels, CI)<br />Disciplined (don’t break/bend the rules)<br />Self sufficient and cross-functional (devs, customers, testers, ops, dbas)<br />Rehearses deployment and roll back.<br />Everybody is responsible for delivery.<br />Everyone can self-service deployments.<br />Continuously improving automation speed (fast build etc.) <br />The team is continuously improving.<br />
    21. 21. Continuous Improvement<br />A self organizing team is like evolution, only with a shorter time span.<br />In biological systems self-organization is a process in which pattern at the global level of a system emerges solely from numerous interactions among the lower-level components of the system. Moreover, the rules specifying interactions among the system's components are executed using only local information, without reference to the global pattern. (Wikipedia)<br />You must improve!<br />Process.<br />Build quality in (don’t waste time on mass inspection).<br />You must become better!<br />Skills.<br />You must learn!<br />Try things you don’t know the outcome from.<br />Fail fast.<br />If it hurts, do it more often, and bring the pain forward.<br />
    22. 22. Robust Applications / Architecture<br />
    23. 23. Open Source == Less bugs<br />
    24. 24. But! That is not my stack!<br />Not all technologiesareeasy to workwithwhenimplementingContinuous Delivery.<br />Whatstackareyou (forced to be) using?<br />Knoweachdependency in anystack<br />Why is it there?<br />Whatdoes it do?<br />Whytheversion?<br />
    25. 25. Nightmare Architects<br />Ivory Tower Architects<br />Strives towards higher levels of abstractions portable across platforms and systems. (Believes there is a single “the Solution” to architecture).<br />Loves boxes and straight arrows.<br />May code frameworks that future projects have to use*<br />Usually won’t listen to neither developers nor users / customers.<br />More detrimental to any Enterprise than they will ever realize.<br />Tell-tales: <br />“Use JSF on all web-applications”.<br />“You haven’t used all eight layers that our guidelines prescribe”.<br />“OK. As long as it is Oracle/IBM”.<br />“Everything has to go through the service bus”.<br />“All applications should follow <title of 500 page guideline document>”.<br />
    26. 26. Ah well. I agreebut…How to handle transition?<br />Gainexpertise (you’rehere, right).<br />Picking up newtechnologies or tools is work.<br />Prove it!<br />Knowhow to handle sceptics.<br />Look for win-winsituations and synergyeffects.<br />Teardown silos. (Communication, collaboration, connectivity).<br />Make it compelling and getpublicity.<br />Improveyour soft skills.<br />Go aroundthem (NB!)<br /><ul><li>Ignore the Irrational
    27. 27. Target the Willing
    28. 28. Harness the Converted
    29. 29. Convince Management</li></li></ul><li>My kind of Architect<br />Pragmatic Architects<br />Codes or are at least highly able to jump into code.<br />Uses foresight, but truly believes that architectures are grown (incrementally) and constantly thinks about the dynamic nature of changing architecture.<br />Accepts that there is no “one solution”.<br />Does not hesitate to change an abstraction if it does not fit.<br />Tell-tales:<br />“The ESB is too slow for handling this load, use the endpoint directly”.<br />“Have you considered a SoftReference there in order to not OOM?”.<br />“Set a Socket Timeout on that endpoint.”.<br />“We use <dependency> <version> because of <esoteric patch>”.<br />“How will the servers handle the load if one node dies?”.<br />
    30. 30. Be Cynical with architecture.<br />Whatever you do not test against, WILL happen.<br />Assume all Integration Points will go down, congest, return corrupt data, incomplete data, be slow, …<br />Assume all low-level connections (pools, concurrency, …) will at sometime fail.<br />Assume your application will die. <br />Know your application’s life cycle, and failure modes<br />Keep environments as equal as possible.<br />Attack environment-based configuration with vigor.<br />
    31. 31. Choose automatable Technology<br />Use an embedded container. (Executable jar with war)<br />No need to deploy, just start and stop. <br />Trivial to setup.<br />Full control over the JVM.<br />Easy to automate.<br />Easy to post-mortem (kill -3 <pid>)<br />Some containers that work:<br />Jetty<br />Netty / Grizzly <br />Glassfish (at least before Oracle)<br />See: https://github.com/bekkopen/jetty-pkg<br />
    32. 32. Understand Application/ Failure modes<br />Traditional (UNIX) application Life cycles<br />start, stop, status, reload <br />Use pidfiles, locks<br />Application-critical life cycles<br />graceful-stop/drain, set-read-only<br />Support inspection of application state (status)<br />health-check, session-status, build version, active feature toggles, configuration<br />Support quick inspection of failed applications<br />thread-dump (kill -3), backup-db, log-backup<br />
    33. 33. Understand (and automate) Application Life Cycles<br />Understand logging concerns<br />Operation/application status events (errors) - AUTOMATE<br />Business events/Audit events (warn)<br />Debugging events (info, debug, …)<br />Keep configuration minimal<br />Separate environment and application configuration*<br />Keep environment configuration to an absolute minimum<br />Know your hardware limitations<br />If one of four balanced nodes die, and the nodes are 50% utilized. The load on a single node will be significant (66% utilized). If another dies the system is on the edge.<br />
    34. 34. Longevity: Impulse and Strain<br />Impulses (Stress)<br />Christmas<br />Getting Slashdotted<br />Annual settlement adding 100 million transactions to a queue<br />Counter: Heavy load tests, Fallback switches<br />Strain (Longevity)<br />Memory leakage and consumption<br />Data growth<br />Dying connections / low level<br />Counter: Timeouts, Fallbacks<br />
    35. 35. Continuous Integration<br />Run against develop branch.<br />Clone builds for long-lived branches.<br />Automate testing (against a production like environment).<br />Unit tests, integration tests, system tests, performance tests, security tests,(functional acceptance tests)…<br />Keep the build fast.<br />Unit tests, integration tests, slow tests.<br />Deploy the latest executable to a repository (Nexus, Artifactory)<br />Visibility (Lava lamps, monitors, air strike alarm…, e-mail)<br />Automate deployment.<br />See: http://martinfowler.com/articles/continuousIntegration.html<br />
    36. 36. Safety measures (Continuously run)<br />Fail fast<br />Build<br />Metrics<br />System tests<br />Environment tests<br />Integration tests<br />Application tests<br />
    37. 37. Metrics<br />What do you want to measure?<br />Test coverage?<br />Lines of code?<br />Code metrics?<br />Checkstyle violations?<br />Do you automate metrics and create reports (e. g. Sonar)?<br />How often do you look at those reports?<br />Alternative: Automate and break the build if you violate your rules.<br />
    38. 38. Continuous Deployment (Push)<br />Push<br />Invoke<br />Invoke<br />Pull<br />Pull/Push<br />
    39. 39. Continuous Deployment (Pull)<br />Pull<br />Push<br />Invoke<br />Invoke<br />Pull<br />Pull/Push<br />
    40. 40. When to push and when to pull?<br />Push:<br />Allows anyone to push.<br />Automated.<br />Caller configures server.<br />Easier?<br />Pull:<br />Initiated by authorized caller.<br />Same artifact is pulled to different environments.<br />Server configures itself.<br />Safer?<br />Vague difference:<br />Push -> Pull: Server requests a push.<br />Pull -> Push: Sshand issue pull command.<br />
    41. 41. Version Control in Teams<br />
    42. 42. What is Version Control?<br />A repository ofcontent.<br />Access to historicaleditionsofeach datum.<br />A log of all changes.<br />A toolthatmanages and tracks different versionsofsoftware or othercontent.<br />
    43. 43. Version Control for Continuous Delivery<br />Keep (almost) everything in version control!<br />WiP must be separated from code in production.<br />Hot-fixes must happen on code in production.<br />Release branches prevent freezes and delays.<br />Avoid big scary merges.<br />
    44. 44. From this …<br />…to this<br />
    45. 45. Work in Progress (WiP)<br />F1<br />F2<br />F3<br />F4<br />F5<br />F6<br />F7<br />F8<br />F9<br />F10<br />Time<br />
    46. 46. We have to separate WiP from production ready code<br />Extra production release<br />Master<br />Initial production version<br />Next production release<br />Next production release<br />Bug!<br />Merge<br />Fix<br />Develop<br />F1<br />F2<br />F3<br />F4<br />F5<br />F6<br />F7<br />F8<br />F9<br />F9<br />Oh no!<br />Release plan 1<br />Release plan 2<br />Release plan 3<br />
    47. 47. Git – Create development branch<br />$ git checkout -b develop<br />Switched to a new branch ”develop"<br />
    48. 48. We have to detach features from the release plans<br />Time<br />Master<br />Initial production version<br />Next production release<br />Next production release<br />Big bug!<br />!<br />Fix<br />Develop<br />F1<br />F4<br />F8<br />Feature branches<br />F2<br />F5<br />F7<br />F10<br />F3<br />F6<br />F9<br />
    49. 49. Git – Feature branches<br />Create:<br />$ git checkout -b <id>_<description><br />Switched to a new branch "<id>_<description>”<br />Merge:<br />$ git checkout develop<br />Switched to branch ’develop'<br />$ git merge --no-ff <id>_<description><br />Updating ea1b82a..05e9557<br />(Summary of changes)<br />$ git push origin develop<br />Delete:<br />$ git branch -d <id>_<description><br />Deleted branch <id>_<description><br />
    50. 50. We have to detach hot-fixes from production and develop<br />Extra production release<br />Extra production release<br />Extra production release<br />Master<br />Initial production version<br />Next production release<br />Next production release<br />Big bug!<br />!<br />!<br />Big bug-fix<br />Hot-fix branches<br />!2<br />!1<br />Develop<br />F8<br />F4<br />F1<br />Feature branches<br />F7<br />F10<br />F5<br />F2<br />F6<br />F9<br />F3<br />
    51. 51. Git – Hot-fix branches<br />Create:<br />$ git checkout -b hot-fix-<version>-<id>_<description><br />Switched to a new branch "hot-fix-<version>-<id>_<description>”<br />Commit:<br />git commit -m ”<message>"<br />[hot-fix-<version>-<id>_<description> abbe5d6] <message><br />5 files changed, 32 insertions(+), 17 deletions(-)<br />
    52. 52. Git – Hot-fix branches -> merge and deploy<br />$ git checkout master<br />Switched to branch 'master'<br />$ git merge --no-ffhot-fix-<version>-<id>_<description><br />Merge made by recursive.<br />(Summary of changes)<br />$ mvn clean release:prepare<br />$ mvn clean release:perform -Dgoals=deploy<br />$ mvn clean deploy<br />$ git checkout develop<br />Switched to branch ’develop'<br />$ git merge --no-ffhot-fix-<version>-<id>_<description><br />Merge made by recursive.<br />(Summary of changes)<br />$ git branch -d hot-fix-<version>-<id>_<description><br />Deleted branch hot-fix-<version>-<id>_<description> (was ff452fe).<br />
    53. 53. Code freezes, testing, preparation … - Release branches<br />Master<br />Initial production version<br />Next production release<br />Next production release<br />Next production release<br />!<br />Fix!<br />Hot-fix branches<br />Release 2<br />Release 3<br />Release 1<br />Release branches<br />Develop<br />F8<br />F4<br />F1<br />Feature branches<br />F7<br />F10<br />F5<br />F2<br />F6<br />F9<br />F3<br />
    54. 54. Git – release branches<br />Create:<br />$ git checkout -b release-<version> develop<br />Switched to a new branch "release-<version>”<br />Commit:<br />git commit -m ”<message>"<br />[release-<version> abbe5d6] <message><br />5 files changed, 32 insertions(+), 17 deletions(-)<br />
    55. 55. Git – release branches -> merge and deploy<br />$ git checkout master<br />Switched to branch 'master'<br />$ git merge --no-ff release-<version><br />Merge made by recursive.<br />(Summary of changes)<br />$ mvn clean release:prepare<br />$ mvn clean release:perform -Dgoals=deploy<br />$ mvn clean deploy<br />$ git checkout develop<br />Switched to branch ’develop'<br />$ git merge --no-ffrelease-<version><br />Merge made by recursive.<br />(Summary of changes)<br />$ git branch -d release-<version><br />Deleted branch release-<version> (was ff452fe).<br />
    56. 56. Feature branching vs. Continuous Integration<br />Feature 1<br />BIG SCARY MERGE<br />Feature 2<br />Feature 1<br />OK<br />Feature 2<br />
    57. 57. Avoiding the Big Scary Merge<br />The problem with continuous integration:<br />Potentially unfinished features in mainline.<br />Hence you can not deploy whenever you want.<br />The Digipost project:<br />13 developers.<br />uses feature branching and releases to production every week.<br />have never had serious merge problems.<br />Keep your features small!<br />Improve the features iteratively.<br />You can check in unfinished features (!)<br />As long as it doesn’t compromise the rest of the system.<br />Use feature toggles.<br />
    58. 58. Feature toggles<br />feature_toggles.properties<br />mail.enabled=true<br />sms.enabled=false<br />send_message.jsp<br /><toggle name=mail.enabled><br /> . mail UI elements<br /></toggle><br />SmsService.java<br />...<br />booleansmsEnabled;<br /> if (smsEnabled) {<br />sendSms();<br /> }<br />...<br />Have a common strategy/framework.<br />Configuration in one place.<br />Toggle in as few places as possible.<br />Avoid polluting your code.<br />
    59. 59. Error handling, Monitoring and health verfication.<br />
    60. 60. GET /application/health<br />bring-shipment-number [OK]<br />bring-messagebroker [OK]<br />bring-freightguide-postal-code [OK]<br />nets-epayment [OK]<br />bring-freightguide [OK]<br />bring-label-generator [OK]<br />bring-label-repository [OK]<br />Version: 1.4.4<br />Build: 0b33b727908<br />Node: <censored>.bd.bring.no<br />
    61. 61. How?<br />package no.bring.booking.operations;<br />public interface HealthCheckable {<br /> Health healthCheck();<br />}<br />
    62. 62. Sample VO<br />package no.bring.booking.operations;<br />public class Health {<br /> final String name;<br /> final Boolean status;<br /> public Health(String name, Boolean status) {<br />this.name = name;<br />this.status = status;<br /> }<br />}<br />
    63. 63. Example implementation<br />class ShipmentNumberClient implements HealthCheckable {<br /> // initialized from environment cfg<br /> private final String healthCheckUri; <br /> // ...<br /> @Override<br /> public Health healthCheck() {<br /> return new Health("bring-shipment-number", isHealthy());<br /> }<br /> private booleanisHealthy() {<br /> try {<br />Client client = HttpClientFactory.createTimingoutClient(2000);<br />WebResourcewebResource = client.resource(healthCheckUri);<br />return fromStatusCode(webResource.head().getStatus()) == OK;<br />} catch (Exception ignored) {<br />return false;<br />}<br />}<br />
    64. 64. Build information<br />Maven: buildnumber-maven-plugin<br />Generates ${buildNumber}<br />Can generate timestamp and other build info<br />Maven: git-commit-id-plugin<br />Generates various git metadata. Such as ${git.commit.id}<br />Date: ${maven.build.timestamp}<br />For another example: http://www.blog.project13.pl/index.php/fun/1174/release-maven-git-commit-id-plugin/<br />
    65. 65. Controlled deploymentZero-down-time redeploy(and roll-back).<br />
    66. 66. Tool Chain For Setting up Architecture<br />System configuration<br />Cloud/VM<br />OS Install<br />Application Service Deployment<br />Scripting<br />Fabric<br />Capistrano<br />MCollective<br />ControlTier<br />Go<br />-----------------<br />Puppet<br />Chef<br />Cfengine<br />-----------------<br />Kickstart<br />Jumpstart<br />Solaris LiveUpdate<br />tftp…<br />
    67. 67. Switch environments<br />
    68. 68. Switch environments<br />
    69. 69. Drain nodes<br />
    70. 70. Drain nodes<br />
    71. 71. Next? A/B?<br />
    72. 72. Migrating databases.<br />
    73. 73. Not impossible, but still difficult<br />Practice, practice, practice.<br />Fail fast.<br />Bring the pain forward.<br />Refactor the DB.<br />Deploy with confidence.<br />
    74. 74. Summary, organizational impact and business value.<br />
    75. 75. 3 pillars of Continuous Delivery<br />Automationof build, test, deployment, database migrations, and infrastructure<br />Practices, such as continuous integration, good configuration management, and testing<br />People; having everyone involved in delivery work together throughout the software delivery lifecycle.<br />
    76. 76. Business Value<br />Reduction of cost (through light-weight infrastructure and simplicity).<br />Reduction of risk (through more automation and practice).<br />But most important:<br />Frequent release of new software functionality.<br />How can we benefit from releasing new functionality monthly, weekly, or even daily?<br />How will our organization and business processes need to change?<br />?<br />
    77. 77. Agile/adaptive maturity<br />Continuous delivery may be the next big step in delivering strategic business value to clients quickly, with lower risk and possibly lower cost. However, it won’t happen unless leadership understands its potential strategic impact and the organizational adaptability necessary to implement it.<br />- Jim Highsmith<br />Many organizations are stuck at Agile 101.<br />the rule-based approach to agile (do this, don’t do that).<br />a necessary first step towards becoming agile, but it’s only a first step.<br />The next step is to embrace change and respond respectively.<br />the entire organization, both delivery teams and management.<br />embrace the process changes required to respond rapidly.<br />engage in the collaboration required between development and operations.<br />embrace an adaptive, exploratory mindset.<br />
    78. 78. References<br />http://continuousdelivery.com/ (Jez Humble)<br />http://pragprog.com/titles/mnee/release-it (Michael T. Nygard)<br />http://martinfowler.com/articles/continuousIntegration.html<br />http://nvie.com/posts/a-successful-git-branching-model/<br />http://progit.org/book/<br />
    79. 79. License & Copyright stuff<br />Images used are either fair use (book covers, Star Wars), public domain or Creative Commons BY-SA licensed.<br />All content in this presentation is Creative Commons BY-SA 2.0. Please credit Ole Christian and Stein Ingeif you decide to use the slides.<br />
    80. 80. Recommended reading<br />
    81. 81.
    82. 82. Stein Inge Morisbak<br />Ole Christian Rynning<br />Developer, Technology<br />Creative leader, Technology<br />+47 909 64 372<br />+47 982 19 347<br />stein.inge.morisbak@BEKK.no<br />ole.chr.rynning@BEKK.no<br />@steinim<br />@olecr<br />http://open.bekk.no/<br />

    ×