Continuous Deployment
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


Continuous Deployment






Total Views
Views on SlideShare
Embed Views



5 Embeds 36 14 9 7 5 1



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • What? Why? Tailor for unique tradeoffs, focusing on risk mitigation.Trans: what is cd? What is CI?
  • Incremental logical step. Coined the term, didn’t invent the concept.(IMVU took it to the next level, but others were doing very similar things before IMVU existed)
  • Ancient concept:Expect it on local dev boxes (php-style)SqueakGenera : An entire OS, locally deployable instantly. Musical Live CodingTrans: So… why?
  • Story? System Chat. Threw away a week of work! Dark launch via CD.Bad Assumptions: Code does what it says Code doesn’t negatively affect other code Code is scalable Code handles edge cases correctly 3rd party API allows for certain behavior
  • “Continuous Deployment” coined August 2007, 20k google hits (phrase is only 4 years old)Feb 2009: Blog post lands
  • Where we were: 2008Handful of companiesZero documentationNo common languageWhat changed?Pattern got a name.People started talking about it.
  • Talk through commit deploy processGit pushGithubPing JenkinsJenkins runs python ./ test canvasRuns through a couple hundred automated tests
  • Does it scale?IMVU: Profitable MMO + Virtual Economy, 50 person technical staffCommit-to-live in about 15 minutes.Massive cluster, massive parallelism. Extensive stats tracking.
  • “Gregory House Theory”Not a theory of CDWaterfallApply the theory on a per-change basis (schemas vsTrans: How to change that curve?
  • Hypothesis commits: What happens if I X?Pay attention to your S/N ratio. Refactoring in separate commits to pull out noise. (Behavior preserving? Behavior changing?)
  • Bank that IVLots of “Lean Startup” and “Lean Thinking” benefits, covered elsewhere.
  • Flickr switchesChrome dev-channel crashes constantly, but I still love it. (WebGL!)Google Labs
  • Website is down! Write tests.“Install the client” test, fear barrier.
  • Nagios alerts are an extension of test coverage.Cluster Immune is a hedge against the cost of big regressions. Does a subset of Nagios alerts with finely tuned parameters.Instant rollback (<15s) is so critical. Human processes (how do I do it? Who does it? What’s a serious regression?)Just go read “Release It!” – Michael NygardCan’t take out a MySQL instance with a bad queryApp works even if search is downIsolation: Think AppEngine.Schemas: Lock down. Review. Try to avoid. Key-value store (NoSQL or YesSQL)
  • Schema changes are high friction; they’re often slow and expensive to deploy: most data stores fight the natural order of Continuous Deployment: small discrete schema changes.Everyone has established practices, patterns and unique situations given choice of database, Code works with schema v0 and v1 (i.e. adding a new row; code explicitly selects the columns it wants and ignores new row)Means you can always step the code and the database back (potentially multiple times, but that takes many steps)
  • Schema changes are computationally expensive and risky
  • Nike method: Just do itHot tub method: Ease into itNuclear option: quit your day job (amazon: switch teams)(Last one’s mostly a joke, but f you look at how the growth of agile methodologies: successful projects and dev environments attract talent; talent came from somewhere!)
  • Putting it all together, examples:Web Startup: forget riskEstablished Service: velocity  IMVU“Big Business”: CD to opt-in customers, daily deploy of baked functionality.Medical: CD to test-environment.Principles apply everywhere.

Continuous Deployment Presentation Transcript

  • 1. Continuous Deployment
    Timothy Fitz
    CTO of Canvas
  • 2. “Continuous integration involves integrating early and often, so as to avoid the pitfalls of "integration hell". The practice aims to reduce timely rework and thus reduce cost and development time.”
  • 3. “Continuous deployment involves deploying early and often, so as to avoid the pitfalls of "deployment hell". The practice aims to reduce timely rework and thus reduce cost and development time.”
  • 4. The Vision
    On every key press
    Run automated tests
    • “Live Coding”
  • Eliminate Waste
    • Deploying code validates assumptions
    Bad assumptions cause waste
    Code built on top is waste
    Designbuilt on top is waste
    Thought built on top is waste
  • 5. The Reality
    Change has risk
    Infrastructure isn’t free
  • 6.
  • 7. At Canvas
    Small CD shop
    5 committers
    Deploy process is “git push”
  • 8. At Scale: IMVU
    MMO + Virtual Economy Etsy
    50+ Technical Staff
  • 9. At Scale: Etsy
  • 10. The Deploy Equation
    Direct Value (DV)
    Information Value (IV)
    Deployment Risk
    When IV + DV > Risk: Deploy!
  • 11. Increase Information Value
    Small commits mean more information earlier
    Implement features implementation-risk-first
    Conscious information gathering
  • 12. Increase Direct Value
    Feature shippable from day 0
    Never blocked on deploy cycle
    Higher velocity
    Lean Thinking
  • 13. Risk=Exposure * Probability * Severity
  • 14. Decrease Exposure
    Dark launch non-frontend changes
    Controlled exposure via feature rollout code
    Expose to staff/QA only
    Expose to opt-in beta testers
    Gradually increase exposure from 1-100%
    Feature-level rollback
  • 15. Decrease Probability
    Automated tests
    Regression / Functional / Integration tests
    Unit tests
    Browser tests / Click tests
    3rd party integration tests
    Manual QA prior to exposing features
    Build code in a deploy mindset
  • 16. Decrease Severity
    Decrease length of degradation
    Production Alerts
    • Cluster Immune System
    • 17. Instant production roll back
    Decrease effects of degradation
    Stability through isolation
    Product level fault tolerance
    Lock down core infrastructure
  • 18. FAQ
    Whatabout shema changes?
    Great, how do I get started?
  • 19. Schema Changes: They hate your code
    Code and schema move in locked steps
    Favor schemaless design
    Minimize classical schema changes
    Offend DBAs with your lack of normalization
    Lightweight/Schemaless databases(“nosql”)
  • 20. Schema Changes: They hate your uptime
    Did I mention schemalessdatabases yet?
    Apply updates to standbys
    Blue/Green cluster setup
  • 21. Great, how do I get started?
    Nike method: Just do it
  • 22. tl;dr
    We’ve come a long way
    We have a long way to go
    IV + DV > Exposure * Probability * Severity.
    Rethink schema changes
    Continous Deployment: Just do it