Continuous
Deployment
Brian Moon
dealnews.com
@brianlmoon
http://brian.moonspot.net/

What is it?
It is the culture of updating production
software when individual new features or bug
fixes are ready for production.
The process will be different from one shop to
another, but the culture is where it counts.

Production Code Updates per Day Q1
15
10
5
0
1/1 1/4 1/7 1/10 1/13 1/16 1/19 1/22 1/25 1/28 1/31 2/3 2/6 2/9 2/12 2/15 2/18 2/21 2/24 2/27 3/1 3/4 3/7 3/10 3/13 3/16 3/19 3/22 3/25
Production Puppet Updates per Day Q1
10
8
6 Also made 43 SQL
Schema changes
4
2
0
1/1 1/4 1/7 1/10 1/13 1/16 1/19 1/22 1/25 1/28 1/31 2/3 2/6 2/9 2/12 2/15 2/18 2/21 2/24 2/27 3/1 3/4 3/7 3/10 3/13 3/16 3/19 3/22 3/25

Who Does This?
• Facebook
• Amazon
• Twitter
• Etsy
• Flickr

What is it not?
• Nightly Builds - nightly builds are not
always ready
• Cowboy Development - rolling code for
the sake of rolling code

Continuous Deployment
Continuous Integration (CI)
Version Control

Developer Developer Developer
Trunk CI of all developer code
Staging CI with production code
Production

Benefits

Ownership
• Developers roll code to production
• Developers have access to production
servers and production data

Agility
• When bugs to occur, it is much quicker to
identify what might have caused it.
• When new business requirements arise,
time to implementation is greatly reduced.

Real Example
• 11:45:47 AM - Editor reports a bug in a critical
tool
• 11:51:29 AM - Senior developer analyzes logs, sees
the bad data
• 14:18:14 PM - Developer merges changes into
trunk. Begins testing on staging.
• 14:35:21 PM - Developer rolls fix. It is live and
working.

Visual Example

Process
Will vary, do what works for you.

Development
• Developers work in their own branch
• They merge changes into trunk when they
are working

Testing
• Code is merged from trunk into a
production checkout on staging
• QA testing and unit testing is performed
there by the developer or via peer review
• When it is determined ready, it is
committed to production branch

Deployment
• The developer responsible for the change
logs in to a production server using a
limited privilege user and updates the code
via a script.
• This central server updates all production
servers with the new code as needed.
• The developer then does a second pass of
testing on production as needed.

Monitoring
• Graph Everything™ and alert on anomalies
• Tools:
• Graphite
• Cacti/Ganglia
• Circonus/Reconnoiter
• Loggly/HopToad
• Nagios
• Keynote/AlertSite/Gomez

Examples
• We send PHP Error counts to Circonus. It
alerts us when more than 1/5/10 per
second occur, escalating as needed
• Nagios monitors load averages of Apache
servers and alerts when it is too high
• AlertSite monitors our page load times
every 15 minutes from 30 locations and
alerts when it slows down

Projects
• Critical people (marketing, management,
etc.) as well as a senior developer discuss
the project
• Once the big picture is clear, senior
developers break down the work into
discrete tasks
• Tasks are completed and rolled as they are
ready, not waiting on all parts to be done.

Real Example
• For SEO purposes, we wanted content that
had expired to redirect to a page of similar
content rather than living forever as an
expired article. But, only for certain
categories of content.

Discrete Tasks
• Update the category maintenance tools,
adding a flag for this behavior
• Change the code that updates our front
end databases, adding this flag
• Change the code that generates a page to
consider this new flag

Summary
• It is a culture, not a strict process
• However, there are common processes that
can be copied
• Give people ownership and they will
flourish
• Monitor after deployment

People Smarter Than Me
• John Allspaw (Etsy, formerly Flickr)
• http://slidesha.re/jaetsy
• http://bit.ly/10deploys
• Theo Schlossnagle (OmniTI)
• http://bit.ly/tsopscareer

Continuous
Deployment
Brian Moon
dealnews.com
@brianlmoon
http://brian.moonspot.net/