0
Continuous Deployment at                     :      A Tale of Two ApproachesRoss Snyderross@etsy.com@beamrider9           ...
A quick primer on
is:The global marketplace we make together.
is:The premier destination for handmade goods,      vintage items, and craft supplies.   simplertimestoys    lacklusterco ...
quick facts:         (as of March 2013)• 22+ million members• 800,000+ active shops• 18+ million items currently for sale•...
Since opening its doors in June 2005, Etsy      has grown virtually non-stop. $1,000  $800  $600  $400  $200    $0        ...
A nice problem to have:    “Our site is sosuccessful, how can we move fast enough tokeep up with demand?”
CONTINUOUSDEPLOYMENT
:The Early Years  (2005 - 2008)
: The Early Years1. Spend significant time writing code
: The Early Years1. Weeks writing code      2. Painful source control merge
: The Early Years1. Weeks writing code2. Painful merge 3. Hand off to someone else to deploy
: The Early Years1. Weeks writing code2. Painful merge3. Hand off to deployers         4. Deploy, site goes down
: The Early Years1. Weeks writing code     4. Deploy, site down2. Painful merge3. Hand off to deployers              5. Rol...
: The Early Years1. Weeks writing code     4. Deploy, site down2. Painful merge          5. Roll back deploy3. Hand off to ...
: The Early Years1. Weeks writing code     4. Deploy, site down2. Painful merge          5. Roll back deploy3. Hand off to ...
: The Early Years            WATERFALL!
: The Early YearsPros:Early Etsy engineersused this releasecycle to bootstrap themarketplace fromnothing.Forever grateful.
: The Early YearsCons:• Large changesets• Infrequent deploys• Weak confidence in deploy success• Significant time spent depl...
: The Early YearsBy late 2008, Etsy is still a startup, but has the deploy process of a much bulkier company.Popularity is...
:Today
: Today1. Small changesets, deployed frequently
: Today1. Small changesets        2. Engineers deploy the site
: TodayAnd not just engineers, but also:• Designers• Product Folks• Upper Management• Board Members• Dogs
: Today1. Small changesets2. Engineers deploy 3. Deploys are fast and near-effortless
: Today1. Small changesets2. Engineers deploy3. Deploys are fast   4. Most changes behind config flags             (safer de...
: Today1. Small changesets   4. Changes behind flags2. Engineers deploy3. Deploys are fast   5. Graphs/metrics to assess de...
: Today1. Small changesets    4. Changes behind flags2. Engineers deploy    5. Copious graphs/metrics3. Deploys are fast6. ...
: Today This isn’t license to break stuff, quickly.Engineer-driven QA and solid unit testing    are integral parts of the p...
: Today1. Small changesets   4. Changes behind flags2. Engineers deploy   5. Copious graphs/metrics3. Deploys are fast   6....
Then:  1. Weeks writing code  2. Painful merge  3. Hand off to deployers  4. Deploy, site down  5. Roll back deploy  6. Fix...
Etsy Deploy Stats: 2012• Deployed to production 6,419 times• On average, 535/month, 25/day• Additional 3,851 config-only de...
Why does it work?
Continuous Deployment Math • N = # of deploys • P = probability of site degradation • S = average severity of degradation ...
Continuous Deployment MathN = # of deploys                  S = avg. severity of degradationP = prob. of degradation      ...
Big Takeaway     Etsy circa 2013 (400+ employees)acts, in some ways, more like a startup than     Etsy circa 2008 (40+ emp...
Continuous Deployment makes possible:     “Continuous Experimentation”   http://etsy.me/continuous-experimentation
Continuous Experimentation   1. Small changes   2. Run experiment (A/B test)   3. Analyze data   4. Re-examine assumptions...
Heard since 2010:“Neat experiment, but this will never scale.” As of 2013, Etsy has  100+ engineers -  still going strong.
Some Etsy CustomizationsDeploying is a first-class feature. Inability todeploy is a P1 incident (same as site down).
Some Etsy CustomizationsWe continuously deploy not just the main Etsy website, but as much as possible:        • Internal ...
Some Etsy Customizations   In the rare case we can’t continuously    deploy, we create alternative tools:   • Database sch...
Some Etsy CustomizationsKeeping deploys fast is paramount and worth the investment in manpower & hardware.
Some Etsy Customizations  Continuous deployment is all about movingforward, sometimes at the expense of the past.Our solut...
Fun Fact: Continuous Deployment is a fantasticrecruitment tool for attracting engineerswho like to move fast and get stuff ...
Learn more:http://codeascraft.etsy.com/Etsy open source (Deployinator, StatsD)http://etsy.github.com/Join the fun:http://w...
Upcoming SlideShare
Loading in...5
×

Ross Snyder, Etsy, SXSW Lean Startup 2013

12,428

Published on

Ross Snyder, Etsy, SXSW Lean Startup 2013

0 Comments
26 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
12,428
On Slideshare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
113
Comments
0
Likes
26
Embeds 0
No embeds

No notes for slide

Transcript of "Ross Snyder, Etsy, SXSW Lean Startup 2013"

  1. 1. Continuous Deployment at : A Tale of Two ApproachesRoss Snyderross@etsy.com@beamrider9 March 9, 2013
  2. 2. A quick primer on
  3. 3. is:The global marketplace we make together.
  4. 4. is:The premier destination for handmade goods, vintage items, and craft supplies. simplertimestoys lacklusterco norwesterseaglass
  5. 5. quick facts: (as of March 2013)• 22+ million members• 800,000+ active shops• 18+ million items currently for sale• 20 cents to list item, 3.5% transaction fee• 400+ employees (majority in Brooklyn)
  6. 6. Since opening its doors in June 2005, Etsy has grown virtually non-stop. $1,000 $800 $600 $400 $200 $0 2005 2006 2007 2008 2009 2010 2011 2012 Gross Merchandise Sales ($MM)
  7. 7. A nice problem to have: “Our site is sosuccessful, how can we move fast enough tokeep up with demand?”
  8. 8. CONTINUOUSDEPLOYMENT
  9. 9. :The Early Years (2005 - 2008)
  10. 10. : The Early Years1. Spend significant time writing code
  11. 11. : The Early Years1. Weeks writing code 2. Painful source control merge
  12. 12. : The Early Years1. Weeks writing code2. Painful merge 3. Hand off to someone else to deploy
  13. 13. : The Early Years1. Weeks writing code2. Painful merge3. Hand off to deployers 4. Deploy, site goes down
  14. 14. : The Early Years1. Weeks writing code 4. Deploy, site down2. Painful merge3. Hand off to deployers 5. Roll back deploy
  15. 15. : The Early Years1. Weeks writing code 4. Deploy, site down2. Painful merge 5. Roll back deploy3. Hand off to deployers 6. Spend hours (days?) fixing bugs
  16. 16. : The Early Years1. Weeks writing code 4. Deploy, site down2. Painful merge 5. Roll back deploy3. Hand off to deployers 6. Fix bugs 7. Go back to step 2
  17. 17. : The Early Years WATERFALL!
  18. 18. : The Early YearsPros:Early Etsy engineersused this releasecycle to bootstrap themarketplace fromnothing.Forever grateful.
  19. 19. : The Early YearsCons:• Large changesets• Infrequent deploys• Weak confidence in deploy success• Significant time spent deploying• Low ability to experiment/iterate/react• Developer stress/unhappiness
  20. 20. : The Early YearsBy late 2008, Etsy is still a startup, but has the deploy process of a much bulkier company.Popularity is on the verge of outpacing capacity.
  21. 21. :Today
  22. 22. : Today1. Small changesets, deployed frequently
  23. 23. : Today1. Small changesets 2. Engineers deploy the site
  24. 24. : TodayAnd not just engineers, but also:• Designers• Product Folks• Upper Management• Board Members• Dogs
  25. 25. : Today1. Small changesets2. Engineers deploy 3. Deploys are fast and near-effortless
  26. 26. : Today1. Small changesets2. Engineers deploy3. Deploys are fast 4. Most changes behind config flags (safer deploys)
  27. 27. : Today1. Small changesets 4. Changes behind flags2. Engineers deploy3. Deploys are fast 5. Graphs/metrics to assess deploy
  28. 28. : Today1. Small changesets 4. Changes behind flags2. Engineers deploy 5. Copious graphs/metrics3. Deploys are fast6. If issues, fix immediately & roll forward
  29. 29. : Today This isn’t license to break stuff, quickly.Engineer-driven QA and solid unit testing are integral parts of the process.
  30. 30. : Today1. Small changesets 4. Changes behind flags2. Engineers deploy 5. Copious graphs/metrics3. Deploys are fast 6. Fix fast & roll forward 7. Repeat 25+ times per day, every day
  31. 31. Then: 1. Weeks writing code 2. Painful merge 3. Hand off to deployers 4. Deploy, site down 5. Roll back deploy 6. Fix bugs, go to step 2Now: 1. Small changesets 2. Engineers deploy 3. Deploys are fast 4. Changes behind flags 5. Copious graphs/metrics 6. Fix fast & roll forward
  32. 32. Etsy Deploy Stats: 2012• Deployed to production 6,419 times• On average, 535/month, 25/day• Additional 3,851 config-only deploys• 196 different people deployed to prod• Nov/Dec 2012: deployed 752 times
  33. 33. Why does it work?
  34. 34. Continuous Deployment Math • N = # of deploys • P = probability of site degradation • S = average severity of degradation • T = time to detect/resolve Expected = N*P*S*T Downtime
  35. 35. Continuous Deployment MathN = # of deploys S = avg. severity of degradationP = prob. of degradation T = time to detect/resolve Before: Now: •N=1 • N = 250 ↑↑↑↑ • P = 0.5 • P = 0.1 ↓ • S = 0.7 • S = 0.05 ↓↓ • T = 100 • T = 5 ↓↓↓ E.D. = 35 E.D. = 6.25 (all numbers completely arbitrary)
  36. 36. Big Takeaway Etsy circa 2013 (400+ employees)acts, in some ways, more like a startup than Etsy circa 2008 (40+ employees).
  37. 37. Continuous Deployment makes possible: “Continuous Experimentation” http://etsy.me/continuous-experimentation
  38. 38. Continuous Experimentation 1. Small changes 2. Run experiment (A/B test) 3. Analyze data 4. Re-examine assumptionsRepeat continuously in pursuit of larger goals.
  39. 39. Heard since 2010:“Neat experiment, but this will never scale.” As of 2013, Etsy has 100+ engineers - still going strong.
  40. 40. Some Etsy CustomizationsDeploying is a first-class feature. Inability todeploy is a P1 incident (same as site down).
  41. 41. Some Etsy CustomizationsWe continuously deploy not just the main Etsy website, but as much as possible: • Internal admin site • API • Big data • Search • Blog • Deployinator itself
  42. 42. Some Etsy Customizations In the rare case we can’t continuously deploy, we create alternative tools: • Database schema changes • PCI-DSS environment (credit cards) We do continuously deploy as much of ourpayment processing as is safe & legal (98%).
  43. 43. Some Etsy CustomizationsKeeping deploys fast is paramount and worth the investment in manpower & hardware.
  44. 44. Some Etsy Customizations Continuous deployment is all about movingforward, sometimes at the expense of the past.Our solution: engineering-wide bug rotation,one day a month, every engineer participates.
  45. 45. Fun Fact: Continuous Deployment is a fantasticrecruitment tool for attracting engineerswho like to move fast and get stuff done.
  46. 46. Learn more:http://codeascraft.etsy.com/Etsy open source (Deployinator, StatsD)http://etsy.github.com/Join the fun:http://www.etsy.com/careers
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×