Development, Deployment & Collaboration at Etsy
•
5 likes•1,634 views
This document discusses development, deployment, and collaboration practices at Etsy. It describes that Etsy deploys changes 50 times per day on average and uses practices like continuous integration, developer VMs, the "Deployinator" tool, monitoring dashboards, on-call rotations, incident response procedures, post-mortem reviews, and knowledge sharing to facilitate frequent deployments in a collaborative environment. The goal is to share practices that work for Etsy and encourage learning and teaching across teams.
More Related Content
What's hot
What's hot (20)
Similar to Development, Deployment & Collaboration at Etsy
Similar to Development, Deployment & Collaboration at Etsy (20)
More from Daniel Schauenberg
Recently uploaded
Recently uploaded (20)
Development, Deployment & Collaboration at Etsy
- 1. Development, Deployment and Collaboration at Etsy Daniel Schauenberg dschauenberg@etsy.com @mrtazz
- 8. avg n > m deploys/ day
- 9. How comfortable are you deploying a change right now?
- 13. “If this is your first day at Etsy, you deploy the site”
- 14. Developer VMs
- 15. @mrtazz Developer VMs • KVM • Every engineer has one • Fully Chef’d with the Etsy Stack • Different sizes and Chef roles
- 19. @mrtazz Continuous Integration • Run set of tests before each deploy • Full QA suite • Princess/Production smoker tests • Try (yup, there is one)
- 21. @mrtazz The Bobs • LXC virtualized hosts • 14/physical hosts • Spread over 3 SSDs • Most of them attached to try
- 24. Deployinator
- 25. @mrtazz Deployinator • 2 Buttons, no ambiguity • Overview of current state of deploy • Links to Logwatcher and Dashboards • Easy to add stacks for new tools to deploy
- 28. Monitoring
- 30. @mrtazz Monitoring • Devs do their feature monitoring • Everybody can access all the graphs • Dashboard All The Things! • Stream All The Logs!
- 34. On Call
- 35. If you are writing code, you are on-call
- 36. @mrtazz On-Call Schedules • ops on-call • dev on-call • payments on-call • support on-call
- 38. @mrtazz Dev On-Call • On-call for 3 days • All developers who are not in another rotation • L1 and L2 escalations • L1 if it’s your first time
- 40. @mrtazz Incident Response • “This graph looks funny” • “Hey I just got paged for elevated error rate after deploys” • “Supergrep is going crazy!!”
- 41. Is the site down?
- 43. #warroom
- 44. @mrtazz #warroom • only outage related conversations • coordinate investigation, communication, countermeasures and monitoring • good place to lurk for new engineers
- 45. Post Mortems
- 46. blameless
- 49. Summary
- 50. @mrtazz Summary • These are things that work for *us* • Culture is an on-going effort • Share everything • Encourage learning/teaching
- 51. @mrtazz Summary • Lunch ’n learns • DC visits • On-call for a day • Bootcamps/Senior rotations
- 53. Questions?
- 54. Development, Deployment and Collaboration at Etsy Daniel Schauenberg dschauenberg@etsy.com