Ross Snyder, Etsy, SXSW Lean Startup 2013
Upcoming SlideShare
Loading in...5
×
 

Ross Snyder, Etsy, SXSW Lean Startup 2013

on

  • 11,798 views

Ross Snyder, Etsy, SXSW Lean Startup 2013

Ross Snyder, Etsy, SXSW Lean Startup 2013

Statistics

Views

Total Views
11,798
Slideshare-icon Views on SlideShare
10,553
Embed Views
1,245

Actions

Likes
21
Downloads
96
Comments
0

16 Embeds 1,245

http://oedb.org 816
https://twitter.com 149
http://domaintest001.com 78
http://www.newsblur.com 49
http://classroom-aid.com 41
http://we.talkspirit.com 39
http://newsblur.com 31
http://flavors.me 22
http://librarians.collected.info 9
http://river.dquarium.com 3
http://dev.oedb.org 2
http://classroomaid.wordpress.com 2
http://mywork.collected.info 1
http://dev.newsblur.com 1
http://we.talkspirit.com.bstalkplus.recette.onu 1
http://tweetedtimes.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Ross Snyder, Etsy, SXSW Lean Startup 2013 Ross Snyder, Etsy, SXSW Lean Startup 2013 Presentation Transcript

    • Continuous Deployment at : A Tale of Two ApproachesRoss Snyderross@etsy.com@beamrider9 March 9, 2013
    • A quick primer on
    • is:The global marketplace we make together.
    • is:The premier destination for handmade goods, vintage items, and craft supplies. simplertimestoys lacklusterco norwesterseaglass
    • quick facts: (as of March 2013)• 22+ million members• 800,000+ active shops• 18+ million items currently for sale• 20 cents to list item, 3.5% transaction fee• 400+ employees (majority in Brooklyn)
    • Since opening its doors in June 2005, Etsy has grown virtually non-stop. $1,000 $800 $600 $400 $200 $0 2005 2006 2007 2008 2009 2010 2011 2012 Gross Merchandise Sales ($MM)
    • A nice problem to have: “Our site is sosuccessful, how can we move fast enough tokeep up with demand?”
    • CONTINUOUSDEPLOYMENT
    • :The Early Years (2005 - 2008)
    • : The Early Years1. Spend significant time writing code
    • : The Early Years1. Weeks writing code 2. Painful source control merge
    • : The Early Years1. Weeks writing code2. Painful merge 3. Hand off to someone else to deploy
    • : The Early Years1. Weeks writing code2. Painful merge3. Hand off to deployers 4. Deploy, site goes down
    • : The Early Years1. Weeks writing code 4. Deploy, site down2. Painful merge3. Hand off to deployers 5. Roll back deploy
    • : The Early Years1. Weeks writing code 4. Deploy, site down2. Painful merge 5. Roll back deploy3. Hand off to deployers 6. Spend hours (days?) fixing bugs
    • : The Early Years1. Weeks writing code 4. Deploy, site down2. Painful merge 5. Roll back deploy3. Hand off to deployers 6. Fix bugs 7. Go back to step 2
    • : The Early Years WATERFALL!
    • : The Early YearsPros:Early Etsy engineersused this releasecycle to bootstrap themarketplace fromnothing.Forever grateful.
    • : The Early YearsCons:• Large changesets• Infrequent deploys• Weak confidence in deploy success• Significant time spent deploying• Low ability to experiment/iterate/react• Developer stress/unhappiness
    • : The Early YearsBy late 2008, Etsy is still a startup, but has the deploy process of a much bulkier company.Popularity is on the verge of outpacing capacity.
    • :Today
    • : Today1. Small changesets, deployed frequently
    • : Today1. Small changesets 2. Engineers deploy the site
    • : TodayAnd not just engineers, but also:• Designers• Product Folks• Upper Management• Board Members• Dogs
    • : Today1. Small changesets2. Engineers deploy 3. Deploys are fast and near-effortless
    • : Today1. Small changesets2. Engineers deploy3. Deploys are fast 4. Most changes behind config flags (safer deploys)
    • : Today1. Small changesets 4. Changes behind flags2. Engineers deploy3. Deploys are fast 5. Graphs/metrics to assess deploy
    • : Today1. Small changesets 4. Changes behind flags2. Engineers deploy 5. Copious graphs/metrics3. Deploys are fast6. If issues, fix immediately & roll forward
    • : Today This isn’t license to break stuff, quickly.Engineer-driven QA and solid unit testing are integral parts of the process.
    • : Today1. Small changesets 4. Changes behind flags2. Engineers deploy 5. Copious graphs/metrics3. Deploys are fast 6. Fix fast & roll forward 7. Repeat 25+ times per day, every day
    • Then: 1. Weeks writing code 2. Painful merge 3. Hand off to deployers 4. Deploy, site down 5. Roll back deploy 6. Fix bugs, go to step 2Now: 1. Small changesets 2. Engineers deploy 3. Deploys are fast 4. Changes behind flags 5. Copious graphs/metrics 6. Fix fast & roll forward
    • Etsy Deploy Stats: 2012• Deployed to production 6,419 times• On average, 535/month, 25/day• Additional 3,851 config-only deploys• 196 different people deployed to prod• Nov/Dec 2012: deployed 752 times
    • Why does it work?
    • Continuous Deployment Math • N = # of deploys • P = probability of site degradation • S = average severity of degradation • T = time to detect/resolve Expected = N*P*S*T Downtime
    • Continuous Deployment MathN = # of deploys S = avg. severity of degradationP = prob. of degradation T = time to detect/resolve Before: Now: •N=1 • N = 250 ↑↑↑↑ • P = 0.5 • P = 0.1 ↓ • S = 0.7 • S = 0.05 ↓↓ • T = 100 • T = 5 ↓↓↓ E.D. = 35 E.D. = 6.25 (all numbers completely arbitrary)
    • Big Takeaway Etsy circa 2013 (400+ employees)acts, in some ways, more like a startup than Etsy circa 2008 (40+ employees).
    • Continuous Deployment makes possible: “Continuous Experimentation” http://etsy.me/continuous-experimentation
    • Continuous Experimentation 1. Small changes 2. Run experiment (A/B test) 3. Analyze data 4. Re-examine assumptionsRepeat continuously in pursuit of larger goals.
    • Heard since 2010:“Neat experiment, but this will never scale.” As of 2013, Etsy has 100+ engineers - still going strong.
    • Some Etsy CustomizationsDeploying is a first-class feature. Inability todeploy is a P1 incident (same as site down).
    • Some Etsy CustomizationsWe continuously deploy not just the main Etsy website, but as much as possible: • Internal admin site • API • Big data • Search • Blog • Deployinator itself
    • Some Etsy Customizations In the rare case we can’t continuously deploy, we create alternative tools: • Database schema changes • PCI-DSS environment (credit cards) We do continuously deploy as much of ourpayment processing as is safe & legal (98%).
    • Some Etsy CustomizationsKeeping deploys fast is paramount and worth the investment in manpower & hardware.
    • Some Etsy Customizations Continuous deployment is all about movingforward, sometimes at the expense of the past.Our solution: engineering-wide bug rotation,one day a month, every engineer participates.
    • Fun Fact: Continuous Deployment is a fantasticrecruitment tool for attracting engineerswho like to move fast and get stuff done.
    • Learn more:http://codeascraft.etsy.com/Etsy open source (Deployinator, StatsD)http://etsy.github.com/Join the fun:http://www.etsy.com/careers