How we keep deployments fast at Etsy presented at Scaleconf 2013

1. Scaling Deployment at Etsy
Daniel Schauenberg
dschauenberg@etsy.com
@mrtazz
Thursday, April 18, 13

2. Thursday, April 18, 13

3. the marketplace we make
together
Thursday, April 18, 13

4. February 2013
•1.49 billion page views
•4,215,169 items sold
•$94.7 million of goods sold
•22+ million members
•800,000+ active shops
http://www.etsy.com/blog/news/2013/etsy-statistics-february-2013-weather-report/ | Items by RockerDollJewellery, ZulamimiLand, codice, 42Things
Thursday, April 18, 13

5. LAMMP
Item by TheBackPackShoppe
Thursday, April 18, 13

6. Item by FrankelPhotos
Monolithic App
Thursday, April 18, 13

7. No Branching
Item by NurseryWallArt
Thursday, April 18, 13

8. Deploy Frequency
Thursday, April 18, 13

9. First Day
Item by flowersandfleurons
Thursday, April 18, 13

10. Thursday, April 18, 13

11. IRC
Thursday, April 18, 13

12. Thursday, April 18, 13

13. DeveloperVMs
•KVM
•Dev version of full Etsy stack
•Chef
•DevTools
Thursday, April 18, 13

14. Thursday, April 18, 13

15. Item by codecards
Thursday, April 18, 13

16. % review -r dschauenberg
Thursday, April 18, 13

17. automatically assignedautomatically assigned
Thursday, April 18, 13

18. Try
Item by CSSDesign
Thursday, April 18, 13

19. Actually ...
Thursday, April 18, 13

20. CI - The Old Way
•~10 buildtest servers, single SSD
•running multiple Jenkins executors
•tests pegging IO
•Jenkins is much better at running single executors
Thursday, April 18, 13

21. The Bobs
Item by Signz
Thursday, April 18, 13

22. The Bobs
•LXC containers on buildtests
•Multiple SSDs
•Labels for heavy/any execution
•One heavy executor per disk
Thursday, April 18, 13

23. CI/Try
•~200 Bobs
•Mostly for try
•Constant monitoring for slow tests
Thursday, April 18, 13

24. push train
Item by decomodwalls
Thursday, April 18, 13

25. #push
•IRC channel to organize push trains
•Join a train if you want to deploy changes
•Schedule is planned via the channel topic
•First in the train is the driver (controls the deploy)
•Opening hours: 7am - 10pm NYC time
Thursday, April 18, 13

26. #push
<prod> kseever* + jameslee | jpaul | DanielConvissor (c)
Thursday, April 18, 13

27. #push
bateman* + krunal* + enorris* | tristan (c) + jameslee (c)
+ jlaster (c) | dawa + corey + sandosh + jklein + magera
+ seth_home + mpascual + nathan | bateman | russp (c)
Thursday, April 18, 13

28. pushbot
•.join
•.in
•.good
•.done
Thursday, April 18, 13

29. pushbot
Thursday, April 18, 13

30. https://github.com/etsy/PushBot
Thursday, April 18, 13

31. Item by EsalonPhotography
Thursday, April 18, 13

32. Deployinator
Thursday, April 18, 13

33. Thursday, April 18, 13

34. Thursday, April 18, 13

35. stale commits
Thursday, April 18, 13

36. version checks
buttons disabledbuttons disabled
Thursday, April 18, 13

37. version checks
Thursday, April 18, 13

38. lock down deploys
Thursday, April 18, 13

39. https://github.com/etsy/deployinator
Thursday, April 18, 13

40. Dashboards
deploy linesdeploy lines
Thursday, April 18, 13

41. https://github.com/etsy/dashboard
Thursday, April 18, 13

42. Supergrep
Thursday, April 18, 13

43. https://github.com/etsy/supergrep
Thursday, April 18, 13

44. Thursday, April 18, 13

45. Summary
•Current setup has scaled to ~150 people
•Constantly trying to improve the speed of deployment
•Find weak parts in the process and make them more
robust/faster
•Bring Dev closer to Prod
•Not being able to deploy has the same status as the site
being down
Thursday, April 18, 13

46. codeascraft.etsy.com
www.etsy.com/codeascraft/talks
etsy.github.com
www.etsy.com/careers
Thursday, April 18, 13

47. Questions?
Thursday, April 18, 13

48. Scaling Deployment at Etsy
Daniel Schauenberg
dschauenberg@etsy.com
@mrtazz
Thursday, April 18, 13