Continuous Deployment - Lean LA
by ashmaurya on Oct 24, 2010
- 2,990 views
A case-study on how I transitioned to Continuous Deployment.
A case-study on how I transitioned to Continuous Deployment.
- Benefits
- How to get started
- When to get started
Accessibility
Categories
Tags
More...Upload Details
Uploaded via SlideShare as Apple Keynote
Usage Rights
© All Rights Reserved
Statistics
- Favorites
- 8
- Downloads
- 98
- Comments
- 0
- Embed Views
- Views on SlideShare
- 2,989
- Total Views
- 2,990
Just to get a sense of who is in the room,
How many people here know what Continuous Deployment is?
And how many people practice Continuous Deployment today?
most of you would probably say:
most of you would probably say:
It’s about having a strong initial vision and then systematically testing it through tightly defined experiments.
It’s NOT.
A Lean Startup is NOT about asking customers what they want, but measuring what they do.
A Lean Startup focuses on those and ignores the rest.
most of you would probably say:
Some of this learning happens during the requirements stage in the form of customer discovery when we’re out talking to customers and figuring and out what to build.
But most of this learning happens only after we get the product into customers’ hands in the form of customer validation.
There is very little learning during development and QA.
Sure we learn about other things, just not about customers.
Some of this learning happens during the requirements stage in the form of customer discovery when we’re out talking to customers and figuring and out what to build.
But most of this learning happens only after we get the product into customers’ hands in the form of customer validation.
There is very little learning during development and QA.
Sure we learn about other things, just not about customers.
Some of this learning happens during the requirements stage in the form of customer discovery when we’re out talking to customers and figuring and out what to build.
But most of this learning happens only after we get the product into customers’ hands in the form of customer validation.
There is very little learning during development and QA.
Sure we learn about other things, just not about customers.
Product Development actually gets in the way of learning about customers.
That is exactly what Continuous Deployment does.
Continuous Deployment shortens the cycle time it takes to build, test, and deploy features.
Shorter cycle times mean we get feedback faster and learn faster.
That is exactly what Continuous Deployment does.
Continuous Deployment shortens the cycle time it takes to build, test, and deploy features.
Shorter cycle times mean we get feedback faster and learn faster.
That is exactly what Continuous Deployment does.
Continuous Deployment shortens the cycle time it takes to build, test, and deploy features.
Shorter cycle times mean we get feedback faster and learn faster.
That is exactly what Continuous Deployment does.
Continuous Deployment shortens the cycle time it takes to build, test, and deploy features.
Shorter cycle times mean we get feedback faster and learn faster.
These shorter cycle times translate to faster feedback and faster learning for the startup.
I’ll describe what my development process looked like before and after continuous deployment.
I’ll talk about how I got started and how I build features now.
We used to spend about a week in development and then QA tested for another 2-3 days before we deployed to customers.
Now we release multiple times a day.
Now we build complete standalone sandboxes for development and QA. Each developer has the complete system on their workstation which gives them a lot more freedom to experiment. Each QA machine also runs on a standalone sandbox which makes it easier for us to do isolated testing as well as scale the testing infrastructure horizontally.
Now a release is triggered automatically every time we commit code. The system is run against a battery of tests and only deployed into production if it passes all the tests. We constantly monitor the production environment and can tell good changes from bad changes quickly and revert a release if we need to.
Our release process currently takes about 20 minutes.
Now a typical release is under 25 lines of code.
Before Continuous Deployment, I had to schedule my week for coding days and customer days. Now I can do both in the same day. I schedule my coding in 2 hour blocks usually early in the day. That leaves the rest of the day open for everything else.
The biggest reason for us was having the feeling of there being no safety net. With a traditional development process, there is a QA cycle before deployment which provides a safety net of time and there is some comfort in sharing testing responsibility with someone else.
- despite our best testing efforts, bugs still crept into production
- bugs got more expensive and harder to fix the longer we waited
- but most important of all, we felt we weren’t learning anything about customers during that time
The answer isn’t taking more time in QA but less through test automation and getting better at detecting and fixing issues in production.
- despite our best testing efforts, bugs still crept into production
- bugs got more expensive and harder to fix the longer we waited
- but most important of all, we felt we weren’t learning anything about customers during that time
The answer isn’t taking more time in QA but less through test automation and getting better at detecting and fixing issues in production.
- despite our best testing efforts, bugs still crept into production
- bugs got more expensive and harder to fix the longer we waited
- but most important of all, we felt we weren’t learning anything about customers during that time
The answer isn’t taking more time in QA but less through test automation and getting better at detecting and fixing issues in production.
- despite our best testing efforts, bugs still crept into production
- bugs got more expensive and harder to fix the longer we waited
- but most important of all, we felt we weren’t learning anything about customers during that time
The answer isn’t taking more time in QA but less through test automation and getting better at detecting and fixing issues in production.
- despite our best testing efforts, bugs still crept into production
- bugs got more expensive and harder to fix the longer we waited
- but most important of all, we felt we weren’t learning anything about customers during that time
The answer isn’t taking more time in QA but less through test automation and getting better at detecting and fixing issues in production.
The first product BoxCloud was built using a release early, release often methodology while CloudFire was built using Lean Startup techniques.
Both products are hybrid web/desktop applications. What I mean by that is they have both a web component and a downloaded client component which runs on mac and windows.
The first product BoxCloud was built using a release early, release often methodology while CloudFire was built using Lean Startup techniques.
Both products are hybrid web/desktop applications. What I mean by that is they have both a web component and a downloaded client component which runs on mac and windows.
In Continuous Deployment, these tests are your only line of defense before deploying code, so you have to take failing tests very seriously. We only deploy a release if it passes all the tests. Otherwise, we stop and fix the tests first.
For me, a small batch is the output of a 2 hour work block. We can not always build a full feature in 2 hours but we have gotten good at deploying features incrementally. We start with non-user facing changes first, like api updates and database changes, before building user-facing changes. Even deploying these changes early greatly help to lower the integration risk of a feature.
Now a typical release is under 25 lines of code.
If something goes wrong here, nothing else matters after that.
We already had a continuous integration server that ran a large collection of unit tests but once we took out the formal QA step, we found ourselves preferring functional tests over unit tests since they were much better at reflecting what users actually did with the system.
We got started simply monitoring the health of production servers using off-the-shelf tools like ganglia and nagios. Over time we built other application and business level monitoring into it.
We did this mostly reactively to production issues. 5Whys.
Continuous Deployment shortens the cycle time it takes to deploy features but how do you make sure you’re actually building what customers want and not simply cranking out new features faster.
Here are some rules we follow.
If you’ve followed a customer discovery process, identified your top 3 problems, and defined a mvp, you do not need to push more features until you have validated your mvp.
This doesn’t mean you stop development, but most of your time should be spent measuring and improving existing features versus chasing after new problems to solve.
From experience, I know this can be a hard rule to enforce and the next rule helps with that.
Here’s how it works: Ideally, a new feature must be pulled or tested with more than one customer for it to show up in the backlog. The number of features in-progress is constrained by the number of developers and so is the number of features waiting for validation. This ensures that you cannot deploy a new feature until a previously deployed feature has been validated.
We use google website optimizer, KISSmetrics, Mixpanel, and homegrown scripts to collect quantitative data. It’s important here too, to focus on the macro and track key metrics over time, like revenue and retention, versus just clicks.
I believe the ideal time is as early in the product development cycle as possible - when you are small and have a few or even no customers.
Continuous Deployment is an incremental process that you have to practice to get really good at. But even adopting simple practices like “coding in small batch sizes” paid off for us very quickly. The biggest benefit we have derived from Continuous Deployment is the ability to integrate Customer Development with Product Development.
The fundamental call to action from Continuous Deployment is to “Ship More Frequently”. Once you take that first step, what you need to do next becomes clearer.
Thank you.