Scaling Deployment at EtsyDaniel Schauenbergdschauenberg@etsy.com@mrtazzThursday, April 18, 13
Thursday, April 18, 13
the marketplace we maketogetherThursday, April 18, 13
February 2013•1.49 billion page views•4,215,169 items sold•$94.7 million of goods sold•22+ million members•800,000+ active...
LAMMPItem by TheBackPackShoppeThursday, April 18, 13
Item by FrankelPhotosMonolithic AppThursday, April 18, 13
No BranchingItem by NurseryWallArtThursday, April 18, 13
Deploy FrequencyThursday, April 18, 13
First DayItem by flowersandfleuronsThursday, April 18, 13
Thursday, April 18, 13
IRCThursday, April 18, 13
Thursday, April 18, 13
DeveloperVMs•KVM•Dev version of full Etsy stack•Chef•DevToolsThursday, April 18, 13
Thursday, April 18, 13
Item by codecardsThursday, April 18, 13
% review -r dschauenbergThursday, April 18, 13
automatically assignedautomatically assignedThursday, April 18, 13
TryItem by CSSDesignThursday, April 18, 13
Actually ...Thursday, April 18, 13
CI - The Old Way•~10 buildtest servers, single SSD•running multiple Jenkins executors•tests pegging IO•Jenkins is much bet...
The BobsItem by SignzThursday, April 18, 13
The Bobs•LXC containers on buildtests•Multiple SSDs•Labels for heavy/any execution•One heavy executor per diskThursday, Ap...
CI/Try•~200 Bobs•Mostly for try•Constant monitoring for slow testsThursday, April 18, 13
push trainItem by decomodwallsThursday, April 18, 13
#push•IRC channel to organize push trains•Join a train if you want to deploy changes•Schedule is planned via the channel t...
#push<prod> kseever* + jameslee | jpaul | DanielConvissor (c)Thursday, April 18, 13
#pushbateman* + krunal* + enorris* | tristan (c) + jameslee (c)+ jlaster (c) | dawa + corey + sandosh + jklein + magera+ s...
pushbot•.join•.in•.good•.doneThursday, April 18, 13
pushbotThursday, April 18, 13
https://github.com/etsy/PushBotThursday, April 18, 13
Item by EsalonPhotographyThursday, April 18, 13
DeployinatorThursday, April 18, 13
Thursday, April 18, 13
Thursday, April 18, 13
stale commitsThursday, April 18, 13
version checksbuttons disabledbuttons disabledThursday, April 18, 13
version checksThursday, April 18, 13
lock down deploysThursday, April 18, 13
https://github.com/etsy/deployinatorThursday, April 18, 13
Dashboardsdeploy linesdeploy linesThursday, April 18, 13
https://github.com/etsy/dashboardThursday, April 18, 13
SupergrepThursday, April 18, 13
https://github.com/etsy/supergrepThursday, April 18, 13
Thursday, April 18, 13
Summary•Current setup has scaled to ~150 people•Constantly trying to improve the speed of deployment•Find weak parts in th...
codeascraft.etsy.comwww.etsy.com/codeascraft/talksetsy.github.comwww.etsy.com/careersThursday, April 18, 13
Questions?Thursday, April 18, 13
Scaling Deployment at EtsyDaniel Schauenbergdschauenberg@etsy.com@mrtazzThursday, April 18, 13
Upcoming SlideShare
Loading in...5
×

Scaling Deployment at Etsy

5,495

Published on

How we keep deployments fast at Etsy presented at Scaleconf 2013

Published in: Technology
0 Comments
12 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,495
On Slideshare
0
From Embeds
0
Number of Embeds
19
Actions
Shares
0
Downloads
31
Comments
0
Likes
12
Embeds 0
No embeds

No notes for slide

Scaling Deployment at Etsy

  1. 1. Scaling Deployment at EtsyDaniel Schauenbergdschauenberg@etsy.com@mrtazzThursday, April 18, 13
  2. 2. Thursday, April 18, 13
  3. 3. the marketplace we maketogetherThursday, April 18, 13
  4. 4. February 2013•1.49 billion page views•4,215,169 items sold•$94.7 million of goods sold•22+ million members•800,000+ active shopshttp://www.etsy.com/blog/news/2013/etsy-statistics-february-2013-weather-report/ | Items by RockerDollJewellery, ZulamimiLand, codice, 42ThingsThursday, April 18, 13
  5. 5. LAMMPItem by TheBackPackShoppeThursday, April 18, 13
  6. 6. Item by FrankelPhotosMonolithic AppThursday, April 18, 13
  7. 7. No BranchingItem by NurseryWallArtThursday, April 18, 13
  8. 8. Deploy FrequencyThursday, April 18, 13
  9. 9. First DayItem by flowersandfleuronsThursday, April 18, 13
  10. 10. Thursday, April 18, 13
  11. 11. IRCThursday, April 18, 13
  12. 12. Thursday, April 18, 13
  13. 13. DeveloperVMs•KVM•Dev version of full Etsy stack•Chef•DevToolsThursday, April 18, 13
  14. 14. Thursday, April 18, 13
  15. 15. Item by codecardsThursday, April 18, 13
  16. 16. % review -r dschauenbergThursday, April 18, 13
  17. 17. automatically assignedautomatically assignedThursday, April 18, 13
  18. 18. TryItem by CSSDesignThursday, April 18, 13
  19. 19. Actually ...Thursday, April 18, 13
  20. 20. CI - The Old Way•~10 buildtest servers, single SSD•running multiple Jenkins executors•tests pegging IO•Jenkins is much better at running single executorsThursday, April 18, 13
  21. 21. The BobsItem by SignzThursday, April 18, 13
  22. 22. The Bobs•LXC containers on buildtests•Multiple SSDs•Labels for heavy/any execution•One heavy executor per diskThursday, April 18, 13
  23. 23. CI/Try•~200 Bobs•Mostly for try•Constant monitoring for slow testsThursday, April 18, 13
  24. 24. push trainItem by decomodwallsThursday, April 18, 13
  25. 25. #push•IRC channel to organize push trains•Join a train if you want to deploy changes•Schedule is planned via the channel topic•First in the train is the driver (controls the deploy)•Opening hours: 7am - 10pm NYC timeThursday, April 18, 13
  26. 26. #push<prod> kseever* + jameslee | jpaul | DanielConvissor (c)Thursday, April 18, 13
  27. 27. #pushbateman* + krunal* + enorris* | tristan (c) + jameslee (c)+ jlaster (c) | dawa + corey + sandosh + jklein + magera+ seth_home + mpascual + nathan | bateman | russp (c)Thursday, April 18, 13
  28. 28. pushbot•.join•.in•.good•.doneThursday, April 18, 13
  29. 29. pushbotThursday, April 18, 13
  30. 30. https://github.com/etsy/PushBotThursday, April 18, 13
  31. 31. Item by EsalonPhotographyThursday, April 18, 13
  32. 32. DeployinatorThursday, April 18, 13
  33. 33. Thursday, April 18, 13
  34. 34. Thursday, April 18, 13
  35. 35. stale commitsThursday, April 18, 13
  36. 36. version checksbuttons disabledbuttons disabledThursday, April 18, 13
  37. 37. version checksThursday, April 18, 13
  38. 38. lock down deploysThursday, April 18, 13
  39. 39. https://github.com/etsy/deployinatorThursday, April 18, 13
  40. 40. Dashboardsdeploy linesdeploy linesThursday, April 18, 13
  41. 41. https://github.com/etsy/dashboardThursday, April 18, 13
  42. 42. SupergrepThursday, April 18, 13
  43. 43. https://github.com/etsy/supergrepThursday, April 18, 13
  44. 44. Thursday, April 18, 13
  45. 45. Summary•Current setup has scaled to ~150 people•Constantly trying to improve the speed of deployment•Find weak parts in the process and make them morerobust/faster•Bring Dev closer to Prod•Not being able to deploy has the same status as the sitebeing downThursday, April 18, 13
  46. 46. codeascraft.etsy.comwww.etsy.com/codeascraft/talksetsy.github.comwww.etsy.com/careersThursday, April 18, 13
  47. 47. Questions?Thursday, April 18, 13
  48. 48. Scaling Deployment at EtsyDaniel Schauenbergdschauenberg@etsy.com@mrtazzThursday, April 18, 13
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×