Download as PDF, PPTX
Uploaded byDaniel Schauenberg
Development, Deployment and Collaboration at Etsy
The document outlines Etsy's practices and culture regarding development, deployment, and collaboration. Key elements include the use of developer virtual machines, continuous integration, a simplified deployment process with the Deployinator tool, and a strong emphasis on monitoring and on-call responsibilities. Additionally, it highlights the importance of learning opportunities and a blameless culture in incident response and post-mortem discussions.
More Related Content
Similar to Development, Deployment and Collaboration at Etsy
Development, Deployment and Collaboration at Etsy
- 1. Development, Deployment and Collaborationat Etsy Daniel Schauenberg dschauenberg@etsy.com @mrtazz
- 3.
- 4.
- 5.
- 6.
- 7.
- 8. avg n >m deploys/ day
- 9. How comfortable are youdeploying a change right now?
- 10.
- 11.
- 13. “If this isyour first day at Etsy, you deploy the site”
- 14.
- 15. Developer VMs • KVM •Every engineer has one • Fully Chef’d with the Etsy Stack • Different sizes and Chef roles @mrtazz
- 17.
- 19. Continuous Integration • Runset of tests before each deploy • Full QA suite • Princess/Production smoker tests • Try (yup, there is one) @mrtazz
- 20.
- 21. The Bobs • LXCvirtualized hosts • 14/physical hosts • Spread over 3 SSDs • Most of them attached to try @mrtazz
- 23.
- 24.
- 25. Deployinator • 2 Buttons,no ambiguity • Overview of current state of deploy • Links to Logwatcher and Dashboards • Easy to add stacks for new tools to deploy @mrtazz
- 26.
- 28.
- 29.
- 30. Monitoring • Devs dotheir feature monitoring • Everybody can access all the graphs • Dashboard All The Things! • Stream All The Logs! @mrtazz
- 34.
- 35. If you arewriting code, you are on-call
- 36. On-Call Schedules • opson-call • dev on-call • payments on-call • support on-call @mrtazz
- 37. Dev On-Call • Scheduledfor 6 months • On-call roughly every 4 weeks for 1 week • L1 and L2 escalations • L1 if it’s your first time @mrtazz
- 38.
- 39. Incident Response • “Thisgraph looks funny” • “Hey I just got paged for elevated error rate after deploys” • “Supergrep is going crazy!!” @mrtazz
- 41.
- 42. #warroom • only outagerelated conversations • coordinate investigation, communication, countermeasures and monitoring • good place to lurk for new engineers @mrtazz
- 43.
- 44.
- 45.
- 46.
- 47.
- 48. Summary • These arethings that work for *us* • Culture is an on-going effort • Share everything • Encourage learning/teaching @mrtazz
- 49. Summary • Lunch ’nlearns • DC visits • On-call for a day • Bootcamps/Senior rotations @mrtazz
- 50.
- 51.
- 52. Development, Deployment and Collaboration atEtsy Daniel Schauenberg dschauenberg@etsy.com