Continuous Deployment


Published on

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • What? Why? Tailor for unique tradeoffs, focusing on risk mitigation.Trans: what is cd? What is CI?
  • Incremental logical step. Coined the term, didn’t invent the concept.(IMVU took it to the next level, but others were doing very similar things before IMVU existed)
  • Ancient concept:Expect it on local dev boxes (php-style)SqueakGenera : An entire OS, locally deployable instantly. Musical Live CodingTrans: So… why?
  • Story? System Chat. Threw away a week of work! Dark launch via CD.Bad Assumptions: Code does what it says Code doesn’t negatively affect other code Code is scalable Code handles edge cases correctly 3rd party API allows for certain behavior
  • “Continuous Deployment” coined August 2007, 20k google hits (phrase is only 4 years old)Feb 2009: Blog post lands
  • Where we were: 2008Handful of companiesZero documentationNo common languageWhat changed?Pattern got a name.People started talking about it.
  • Talk through commit deploy processGit pushGithubPing JenkinsJenkins runs python ./ test canvasRuns through a couple hundred automated tests
  • Does it scale?IMVU: Profitable MMO + Virtual Economy, 50 person technical staffCommit-to-live in about 15 minutes.Massive cluster, massive parallelism. Extensive stats tracking.
  • “Gregory House Theory”Not a theory of CDWaterfallApply the theory on a per-change basis (schemas vsTrans: How to change that curve?
  • Hypothesis commits: What happens if I X?Pay attention to your S/N ratio. Refactoring in separate commits to pull out noise. (Behavior preserving? Behavior changing?)
  • Bank that IVLots of “Lean Startup” and “Lean Thinking” benefits, covered elsewhere.
  • Flickr switchesChrome dev-channel crashes constantly, but I still love it. (WebGL!)Google Labs
  • Website is down! Write tests.“Install the client” test, fear barrier.
  • Nagios alerts are an extension of test coverage.Cluster Immune is a hedge against the cost of big regressions. Does a subset of Nagios alerts with finely tuned parameters.Instant rollback (<15s) is so critical. Human processes (how do I do it? Who does it? What’s a serious regression?)Just go read “Release It!” – Michael NygardCan’t take out a MySQL instance with a bad queryApp works even if search is downIsolation: Think AppEngine.Schemas: Lock down. Review. Try to avoid. Key-value store (NoSQL or YesSQL)
  • Schema changes are high friction; they’re often slow and expensive to deploy: most data stores fight the natural order of Continuous Deployment: small discrete schema changes.Everyone has established practices, patterns and unique situations given choice of database, Code works with schema v0 and v1 (i.e. adding a new row; code explicitly selects the columns it wants and ignores new row)Means you can always step the code and the database back (potentially multiple times, but that takes many steps)
  • Schema changes are computationally expensive and risky
  • Nike method: Just do itHot tub method: Ease into itNuclear option: quit your day job (amazon: switch teams)(Last one’s mostly a joke, but f you look at how the growth of agile methodologies: successful projects and dev environments attract talent; talent came from somewhere!)
  • Putting it all together, examples:Web Startup: forget riskEstablished Service: velocity  IMVU“Big Business”: CD to opt-in customers, daily deploy of baked functionality.Medical: CD to test-environment.Principles apply everywhere.
  • Continuous Deployment

    1. 1. Continuous Deployment<br />Timothy Fitz<br />CTO of Canvas<br />
    2. 2. “Continuous integration involves integrating early and often, so as to avoid the pitfalls of "integration hell". The practice aims to reduce timely rework and thus reduce cost and development time.”<br />
    3. 3. “Continuous deployment involves deploying early and often, so as to avoid the pitfalls of "deployment hell". The practice aims to reduce timely rework and thus reduce cost and development time.”<br />
    4. 4. The Vision<br />On every key press<br />Compile<br />Run automated tests<br />Deploy<br /><ul><li>“Live Coding”</li></li></ul><li>Eliminate Waste<br /><ul><li>Deploying code validates assumptions</li></ul>Bad assumptions cause waste<br />Code built on top is waste<br />Designbuilt on top is waste<br />Thought built on top is waste<br />
    5. 5. The Reality<br />Change has risk<br />Infrastructure isn’t free<br />
    6. 6.
    7. 7. At Canvas<br />Small CD shop<br />5 committers<br />Deploy process is “git push”<br />
    8. 8. At Scale: IMVU<br />Profitable<br />MMO + Virtual Economy Etsy<br />50+ Technical Staff<br />
    9. 9. At Scale: Etsy<br />
    10. 10. The Deploy Equation<br />Direct Value (DV)<br />Information Value (IV)<br />Deployment Risk<br />When IV + DV > Risk: Deploy!<br />
    11. 11. Increase Information Value<br />Small commits mean more information earlier<br />Implement features implementation-risk-first<br />Conscious information gathering<br />
    12. 12. Increase Direct Value<br />Feature shippable from day 0<br />Never blocked on deploy cycle<br />Higher velocity<br />Lean Thinking<br />
    13. 13. Risk=Exposure * Probability * Severity<br />
    14. 14. Decrease Exposure<br />Dark launch non-frontend changes<br />Controlled exposure via feature rollout code<br />Expose to staff/QA only<br />Expose to opt-in beta testers<br />Gradually increase exposure from 1-100%<br />Feature-level rollback<br />
    15. 15. Decrease Probability<br />Automated tests<br />Regression / Functional / Integration tests<br />Unit tests<br />Browser tests / Click tests<br />3rd party integration tests<br />Manual QA prior to exposing features<br />Build code in a deploy mindset<br />
    16. 16. Decrease Severity <br />Decrease length of degradation<br />Production Alerts<br /><ul><li>Cluster Immune System
    17. 17. Instant production roll back</li></ul>Decrease effects of degradation<br />Stability through isolation<br />Product level fault tolerance<br />Lock down core infrastructure<br />
    18. 18. FAQ<br />Whatabout shema changes?<br />Great, how do I get started?<br />
    19. 19. Schema Changes: They hate your code<br />Code and schema move in locked steps<br />Favor schemaless design<br />Minimize classical schema changes<br />Offend DBAs with your lack of normalization<br />Lightweight/Schemaless databases(“nosql”)<br />
    20. 20. Schema Changes: They hate your uptime<br />Did I mention schemalessdatabases yet?<br />Apply updates to standbys<br />Blue/Green cluster setup<br />
    21. 21. Great, how do I get started?<br />Nike method: Just do it<br />
    22. 22. tl;dr<br />We’ve come a long way<br />We have a long way to go<br />IV + DV > Exposure * Probability * Severity.<br />Rethink schema changes<br />Continous Deployment: Just do it<br />Questions?<br />