Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deploy, Collaborate and Listen

872 views

Published on

At Etsy we are continuously deploying software with about 50 deploys per day on average. We have about 150 engineers that work together to add features, improve the site, solve problems, and figure out outages. We are constantly working on improving ways to collaborate and have the time to invest into research and new projects instead of fire fighting. However this hasn't been like this from the beginning. In the Dark Ages of Etsy we had a vastly different software architecture, a myriad of silos with a less than ideal amount of communication and a not so stable site. So what happened? How did we fix this? We will briefly go over how we changed the architecture and culture to make site operations and stability better. After that we will go into detail about what it meant for us to maintain that culture of collaboration and trust and things we learned from that. And finally take a look at the state of things in the present and give a comprehensive picture of how we arrived here and where to go from there.

Published in: Technology
  • Be the first to comment

Deploy, Collaborate and Listen

  1. 1. DEPLOY COLLABORATE & LISTEN Daniel Schauenberg d@etsy.com @mrtazz
  2. 2. ETSY
  3. 3. THE MONOLITH
  4. 4. LAMP
  5. 5. HOW COMFORTABLE ARE YOU DEPLOYING A CHANGE RIGHT NOW
  6. 6. MTTR TRUMPS MTBF
  7. 7. IF THIS IS YOUR FIRST DAY AT ETSY YOU DEPLOY THE SITE
  8. 8. THE DARK PAST
  9. 9.
  10. 10. - HINDSIGHT IS 20/20 - THERE WOULD BE NO ETSY - I WASN'T AROUND FOR THIS (the grain of salt disclaimer)
  11. 11. DARK LESS FUN
  12. 12. ARCHITECTURE OVERVIEW - UBUNTU - POSTGRESQL - LIGHTTPD - PHP/PYTHON
  13. 13. SINGLE BIG DATABASE
  14. 14. BUSINESS LOGIC IN STORED PROCEDURES
  15. 15. SILOS
  16. 16. DEV ! DBA ! OPS
  17. 17. SPROUTER
  18. 18. A SOFTWARE MANIFESTATION OF SILOS
  19. 19. SITE UPTIME WASN'T GOOD
  20. 20. MORE UPTIME > LESS UPTIME
  21. 21. Trust THE PEOPLE
  22. 22. SPROUTER
  23. 23. NO MORE MAINFRAME MASTER
  24. 24. HORIZONTAL SCALING (A SINGLE BOX ONLY SCALES SO FAR)
  25. 25. MASTER-MASTER REPLICATED MYSQL SHARDS
  26. 26. FLICKR DNA
  27. 27. DEPLOYINATOR
  28. 28. DEPLOY != RELEASE
  29. 29. CONFIG FLAGS
  30. 30. if Feature::isEnabled($feature) { // new hawtness } else { // nothing to see here }
  31. 31. METRICS!
  32. 32. STATSD "HOW DO YOU KNOW THIS WORKS IN PRODUCTION?"
  33. 33. CHEF
  34. 34. <3 CHEF
  35. 35. DEVELOPER VMS - KVM - EVERY ENGINEER HAS ONE - FULLY CHEF’D WITH THE ETSY STACK - DIFFERENT SIZES AND CHEF ROLES
  36. 36. Try
  37. 37. BLAMELESS POST MORTEMS
  38. 38. OVERHAULS & ITERATIONS
  39. 39. CULTURE & TOOLS
  40. 40. HUMANS ARE AWESOME
  41. 41. NOBODY COMES TO WORK TO DO A BAD JOB
  42. 42. Trust YOUR CO-WORKERS
  43. 43. THERE IS A LOT OF KNOWLEDGE IN YOUR ENGINEERING TEAM
  44. 44. DEPLOY (AS OFTEN AS IT MAKES SENSE)
  45. 45. COLLABORATE (EVEN IF YOU THINK YOU DON'T HAVE TO)
  46. 46. LISTEN (TO PROBLEMS AND EXPERIENCES OF YOUR COWORKERS)
  47. 47. CODEASCRAFT.COM ETSY.COM/CODEASCRAFT/TALKS ETSY.COM/CAREERS
  48. 48. THANK YOU!
  49. 49. DEPLOY COLLABORATE & LISTEN Daniel Schauenberg d@etsy.com @mrtazz

×