Etsy
Deployment
continuous
at
@premshreewebcamp zagreb 2015
1.5 million active sellers
21.7 million active buyers
$1.93 billion GMS (2014)
757 employees
Premshree Pillai
Sr. Engineer, Etsy
bok
principles
q & a
tooling + culture
principles
just ship
enable innovation
enable innovation
(you are creating products, not code)
optimize for purpose
optimize for purpose
optimize for autonomy
optimize for mastery
experimentation
experimentation
A/B testing
quick iteration
quick iteration
(improving products/features)
quick iteration
(fail fast > stagnant code)
continuous improvement
low MTTR
tooling
continuous deployment
continuous delivery
/
commit
commit build
commit build tests
commit build tests user tests
commit build tests user tests release
commit build tests user tests release
commit build tests user tests release
frequent check-ins
frequent check-ins
(to master!)
frequent check-ins
branching in code
Feature
Feature
github.com/etsy/feature
$server_config['my_feature'] = [	
'enabled' => 'on'	
];
$server_config['my_feature'] = [	
'enabled' => 'off'	
];
$server_config['my_feature'] = [	
'enabled' => 1	
];
$server_config['my_feature'] = [	
'enabled' => 1,	
'bucketing' => 'user'	
];
if (Feature::isEnabled('my_feature') {	
!
}
if (Feature::isEnabledFor('my_feature', $user) {	
!
}
Experimentation
Experimentation
A/B testing
$server_config['my_feature'] = [	
'enabled' => 'on'	
];
$server_config['my_feature'] = [	
'enabled' => [	
'layout1' => 1,	
'layout2' => 3,	
'layout3' => 3	
]	
];
if (Feature::isEnabled('my_feature')) {	
switch (Feature::variant('my_feature')) {	
case 'layout1':	
// layout1 code	
break;	
case 'layout2':	
// layout2 code	
break;	
}	
}
continuous integration
continuous integration
(and automated tests)
keep the build green
ready to release
ready to release
anytime
commit build tests user tests release
try
Try
Try
(before you commit)
TryLib
TryLib
github.com/etsy/TryLib
commit build tests user tests release
try deployinator
Deployinator
Deployinator
github.com/etsy/deployinator
commit build tests user tests release
try deployinator ci/jenkins
commit build tests user tests release
try deployinator ci/jenkins manual tests
commit build tests user tests release
try deployinator ci/jenkins manual tests deploy
anybody can push
http://www.flickr.com/photos/zsqr0000/5269002895
be anxious
… but do not fear
dev
/
ops
♥dev
/
ops
dark changes
template changes
css tweaks
unreferenced code
config pushes
$server_config['my_feature'] = [	
'enabled' => 1,	
'bucketing' => 'user'	
];
$server_config['my_feature'] = [	
'enabled' => 50,	
'bucketing' => 'user'	
];
push train
premshree> .join
john
john + premshree
premshree>
john + premshree
premshree> .good
john + premshree
premshree> .bueno
john + premshree*
premshree>
john + premshree*
sally> .join
john + premshree* | sally
sally>
push train
(with PushBot)
push train
github.com/etsy/pushbot
commit build tests user tests release
commit build tests user tests princess prod
post-deploy
post-deploy
(gaining confidence)
supergrep
supergrep
github.com/etsy/supergrep
dashboards
dashboards
(deploy)
push
dashboards
(app)
StatsD + Graphite
StatsD + Graphite
github.com/etsy/statsd
summary
pre-flight check
ready to push
ready to deploy
testing/verification
confidence
pre-flight check
ready to push
ready to deploy
testing/verification
confidence
try/git hooks
push train/irc
deployinator
user testing
supergrep/dashboards
https://www.flickr.com/photos/saschaaa/152502539/
poka-yoke
ポカヨケ
culture
assume best intentions
cultivate empathy
open
failure is an option
failure is an opportunity
post-mortems
blameless post-mortems
+ remediation
morgue
morgue
github.com/etsy/morgue
mixer
mixer
github.com/etsy/mixer
culture + tooling
deploy
fail ok
post-mortem
remediation
stimulii
deploys
30+ deploys
30+ config deploys
codeascraft.com
fin
hvala.
questions?
https://joind.in/15234 @premshree

Continuous Deployment at Etsy