At Etsy our approach to development is to make small, incremental, continuous changes to the site. Deploying code to production anywhere between 20 to 40 times a day requires not only technical tooling, but also a culture that allows for and encourages innovation, confidence backed by actual data and comprehensive tests, and blameless post-mortems that allow for a feedback loop we can learn from and improve. Learn about the tools we have, and how we foster an environment where we make this possible.

1. Scale
Deployment
continuous
at
@premshree

3. 1.6 million active sellers
24 million active buyers
$2.39 billion GMS (2015)
819 employees

4. Premshree Pillai
Sr. Engineer, Etsy

5. tere!

6. principles
q & a
tooling + culture

7. principles

8. just ship

9. enable innovation

10. enable innovation
(you are creating products, not code)

11. optimize for purpose

12. optimize for purpose

13. optimize for autonomy

14. optimize for mastery

15. experimentation

16. experimentation
A/B testing

17. quick iteration

18. quick iteration
(improving products/features)

19. quick iteration
(fail fast > stagnant code)

20. continuous improvement

21. low MTTR

22. tooling

23. continuous deployment
continuous delivery
/

24. commit

25. commit build

26. commit build tests

27. commit build tests user tests

28. commit build tests user tests release

29. commit build tests user tests release

30. commit build tests user tests release

31. frequent check-ins

32. frequent check-ins
(to master!)

33. frequent check-ins
branching in code

34. Feature

35. Feature
github.com/etsy/feature

36. $server_config['my_feature'] = [
'enabled' => 'on'
];

37. $server_config['my_feature'] = [
'enabled' => 'off'
];

38. $server_config['my_feature'] = [
'enabled' => 1
];

39. $server_config['my_feature'] = [
'enabled' => 1,
'bucketing' => 'user'
];

40. if (Feature::isEnabled('my_feature') {
!
}

41. if (Feature::isEnabledFor('my_feature', $user) {
!
}

42. Experimentation

43. Experimentation
A/B testing

44. $server_config['my_feature'] = [
'enabled' => 'on'
];

45. $server_config['my_feature'] = [
'enabled' => [
'layout1' => 1,
'layout2' => 3,
'layout3' => 3
]
];

46. if (Feature::isEnabled('my_feature')) {
switch (Feature::variant('my_feature')) {
case 'layout1':
// layout1 code
break;
case 'layout2':
// layout2 code
break;
}
}

47. continuous integration

48. continuous integration
(and automated tests)

49. keep the build green

51. ready to release

52. ready to release
anytime

53. commit build tests user tests release
try

54. Try

55. Try
(before you commit)

56. TryLib

57. TryLib
github.com/etsy/TryLib

59. commit build tests user tests release
try deployinator

60. Deployinator

61. Deployinator
github.com/etsy/deployinator

63. commit build tests user tests release
try deployinator ci/jenkins

65. commit build tests user tests release
try deployinator ci/jenkins manual tests

66. commit build tests user tests release
try deployinator ci/jenkins manual tests deploy

69. commit build tests user tests release

70. commit build tests user tests princess prod

71. anybody can push

72. http://www.flickr.com/photos/zsqr0000/5269002895

73. be anxious

74. … but do not fear

75. dev
/
ops

76. ♥dev
/
ops

77. dark changes

78. template changes
css tweaks
unreferenced code

80. config pushes

81. $server_config['my_feature'] = [
'enabled' => 1,
'bucketing' => 'user'
];

82. $server_config['my_feature'] = [
'enabled' => 50,
'bucketing' => 'user'
];

83. push train

86. premshree> .join
john

87. john + premshree
premshree>

88. john + premshree
premshree> .good

89. john + premshree
premshree> .bueno

90. john + premshree*
premshree>

91. john + premshree*
sally> .join

92. john + premshree* | sally
sally>

93. push train
(with PushBot)

94. push train
github.com/etsy/pushbot

95. post-deploy

96. post-deploy
(gaining confidence)

97. supergrep

98. supergrep
github.com/etsy/supergrep

101. dashboards

102. dashboards
(deploy)

105. push

106. dashboards
(app)

108. StatsD + Graphite

109. StatsD + Graphite
github.com/etsy/statsd

110. summary

111. pre-flight check
ready to push
ready to deploy
testing/verification
confidence

112. pre-flight check
ready to push
ready to deploy
testing/verification
confidence
try/git hooks
push train/irc
deployinator
user testing
supergrep/dashboards

113. https://www.flickr.com/photos/saschaaa/152502539/

114. poka-yoke
ポカヨケ

115. culture

117. assume best intentions

118. cultivate empathy

119. open

120. failure is an option

121. failure is an option
(but not our intention)

122. failure is an option
(we strongly want NOT to fail)

123. failure is an opportunity

124. post-mortems

125. blameless post-mortems

126. + remediation

127. morgue

130. morgue
github.com/etsy/morgue

132. mixer

133. mixer
github.com/etsy/mixer

134. culture + tooling
deploy
fail ok
post-mortem
remediation
stimulii

136. deploys

137. 30+ deploys

138. 30+ config deploys

139. codeascraft.com

140. fin

141. aitäh

142. questions?
@premshreepolkadotsocks.net