SlideShare a Scribd company logo

Ross Snyder, Etsy, SXSW Lean Startup 2013

Ross Snyder, Etsy, SXSW Lean Startup 2013

1 of 46
Download to read offline
Continuous Deployment at
                     :
      A Tale of Two Approaches

Ross Snyder
ross@etsy.com
@beamrider9            March 9, 2013
A quick primer on
is:
The global marketplace we make together.
is:
The premier destination for handmade goods,
      vintage items, and craft supplies.




   simplertimestoys    lacklusterco   norwesterseaglass
quick facts:         (as of March 2013)



• 22+ million members

• 800,000+ active shops

• 18+ million items currently for sale

• 20 cents to list item, 3.5% transaction fee

• 400+ employees (majority in Brooklyn)
Since opening its doors in June 2005, Etsy
      has grown virtually non-stop.
 $1,000


  $800


  $600


  $400


  $200


    $0
          2005   2006   2007   2008   2009   2010   2011   2012


            Gross Merchandise Sales ($MM)

Recommended

Yuri Warczynski “Claiming the mantle of Julius Caesar,” or how C-management ...
Yuri Warczynski  “Claiming the mantle of Julius Caesar,” or how C-management ...Yuri Warczynski  “Claiming the mantle of Julius Caesar,” or how C-management ...
Yuri Warczynski “Claiming the mantle of Julius Caesar,” or how C-management ...Lviv Startup Club
 
Y Combinator Startup Class #7 : How to Build Products Users Love (Part 1)
Y Combinator Startup Class #7 : How to Build Products Users Love (Part 1)Y Combinator Startup Class #7 : How to Build Products Users Love (Part 1)
Y Combinator Startup Class #7 : How to Build Products Users Love (Part 1)Fabien Grenet
 
Five Reports You Aren't Running & Why You Should Start Today - State Bar of T...
Five Reports You Aren't Running & Why You Should Start Today - State Bar of T...Five Reports You Aren't Running & Why You Should Start Today - State Bar of T...
Five Reports You Aren't Running & Why You Should Start Today - State Bar of T...Haley Odom Ackerman
 
10 Tactics for Building an Optimization Culture
10 Tactics for Building an Optimization Culture10 Tactics for Building an Optimization Culture
10 Tactics for Building an Optimization CultureOptimizely
 
Chaining the Beast - Testing Wonder Applications in the Real World
Chaining the Beast - Testing Wonder Applications in the Real WorldChaining the Beast - Testing Wonder Applications in the Real World
Chaining the Beast - Testing Wonder Applications in the Real WorldWO Community
 
Top 15 reasons to choose qa testing as career
Top 15 reasons to choose qa testing as career Top 15 reasons to choose qa testing as career
Top 15 reasons to choose qa testing as career JanBask Training
 
A no bullshit introduction to Hebel
A no bullshit introduction to HebelA no bullshit introduction to Hebel
A no bullshit introduction to HebelFranz Enzenhofer
 

More Related Content

What's hot

DevOps Traction - Continuous Lifecycle 2016 London
DevOps Traction - Continuous Lifecycle 2016 LondonDevOps Traction - Continuous Lifecycle 2016 London
DevOps Traction - Continuous Lifecycle 2016 LondonDevOps Ltd.
 
BoS2015 Jeff Szczepanski – COO, Stack Exchange - Stack Overflow. Scaling a Te...
BoS2015 Jeff Szczepanski – COO, Stack Exchange - Stack Overflow. Scaling a Te...BoS2015 Jeff Szczepanski – COO, Stack Exchange - Stack Overflow. Scaling a Te...
BoS2015 Jeff Szczepanski – COO, Stack Exchange - Stack Overflow. Scaling a Te...Business of Software Conference
 
SAMPLE SIZE – The indispensable A/B test calculation that you’re not making
SAMPLE SIZE – The indispensable A/B test calculation that you’re not makingSAMPLE SIZE – The indispensable A/B test calculation that you’re not making
SAMPLE SIZE – The indispensable A/B test calculation that you’re not makingZack Notes
 
Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness
Tis The Season: Load Testing Tips and Checklist for Retail Seasonal ReadinessTis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness
Tis The Season: Load Testing Tips and Checklist for Retail Seasonal ReadinessSOASTA
 
Competency Center: retaining experts, growing expertise, maximizing income (O...
Competency Center: retaining experts, growing expertise, maximizing income (O...Competency Center: retaining experts, growing expertise, maximizing income (O...
Competency Center: retaining experts, growing expertise, maximizing income (O...Valentin Kropov
 
A/B Testing and the Infinite Monkey Theory
A/B Testing and the Infinite Monkey TheoryA/B Testing and the Infinite Monkey Theory
A/B Testing and the Infinite Monkey TheoryUseItBetter
 
15 cases of testing with data & psychology #CRO
15 cases of testing with data & psychology #CRO15 cases of testing with data & psychology #CRO
15 cases of testing with data & psychology #CROOnline Dialogue
 
Testing Small Agile Projects from Agile Vancouver Quality Conference 2014
Testing Small Agile Projects from Agile Vancouver Quality Conference 2014Testing Small Agile Projects from Agile Vancouver Quality Conference 2014
Testing Small Agile Projects from Agile Vancouver Quality Conference 2014Lanette Creamer
 
The Hard life of Agile Coach Project in a ruin
The Hard life of Agile Coach Project in a ruinThe Hard life of Agile Coach Project in a ruin
The Hard life of Agile Coach Project in a ruinJakub Drzazga
 
When Support Calls
When Support CallsWhen Support Calls
When Support CallsJames Thomas
 
[HCMC STC Jan 2015] Choosing The Best Of The Plan-Driven And Agile Developmen...
[HCMC STC Jan 2015] Choosing The Best Of The Plan-Driven And Agile Developmen...[HCMC STC Jan 2015] Choosing The Best Of The Plan-Driven And Agile Developmen...
[HCMC STC Jan 2015] Choosing The Best Of The Plan-Driven And Agile Developmen...Ho Chi Minh City Software Testing Club
 
[HCMC STC Jan 2015] Developing an Offshore Context-Driven Testing Team
[HCMC STC Jan 2015] Developing an Offshore Context-Driven Testing Team[HCMC STC Jan 2015] Developing an Offshore Context-Driven Testing Team
[HCMC STC Jan 2015] Developing an Offshore Context-Driven Testing TeamHo Chi Minh City Software Testing Club
 
Warshunckiy- questions to assess project maturity
Warshunckiy- questions to assess  project maturityWarshunckiy- questions to assess  project maturity
Warshunckiy- questions to assess project maturityLviv Startup Club
 
Let's Build a Product Development Organization!
Let's Build a Product Development Organization!Let's Build a Product Development Organization!
Let's Build a Product Development Organization!Kevin Goldsmith
 
Pavel Kamyshov "Team Health Check" Kyiv PM Club
Pavel Kamyshov "Team Health Check" Kyiv PM ClubPavel Kamyshov "Team Health Check" Kyiv PM Club
Pavel Kamyshov "Team Health Check" Kyiv PM ClubLviv Startup Club
 
Value Metric - как использовать бизнес-метрики в agile-проектах
Value Metric  - как использовать бизнес-метрики в agile-проектахValue Metric  - как использовать бизнес-метрики в agile-проектах
Value Metric - как использовать бизнес-метрики в agile-проектахAlexey Pikulev
 
Developing your Developers: Constructing Career Paths for your Technologists ...
Developing your Developers: Constructing Career Paths for your Technologists ...Developing your Developers: Constructing Career Paths for your Technologists ...
Developing your Developers: Constructing Career Paths for your Technologists ...Kevin Goldsmith
 
A/B Testing - How to Break Things and Fail Fast...without Breaking Things
A/B Testing - How to Break Things and Fail Fast...without Breaking ThingsA/B Testing - How to Break Things and Fail Fast...without Breaking Things
A/B Testing - How to Break Things and Fail Fast...without Breaking ThingsMagnolia
 

What's hot (20)

DevOps Traction - Continuous Lifecycle 2016 London
DevOps Traction - Continuous Lifecycle 2016 LondonDevOps Traction - Continuous Lifecycle 2016 London
DevOps Traction - Continuous Lifecycle 2016 London
 
BoS2015 Jeff Szczepanski – COO, Stack Exchange - Stack Overflow. Scaling a Te...
BoS2015 Jeff Szczepanski – COO, Stack Exchange - Stack Overflow. Scaling a Te...BoS2015 Jeff Szczepanski – COO, Stack Exchange - Stack Overflow. Scaling a Te...
BoS2015 Jeff Szczepanski – COO, Stack Exchange - Stack Overflow. Scaling a Te...
 
SAMPLE SIZE – The indispensable A/B test calculation that you’re not making
SAMPLE SIZE – The indispensable A/B test calculation that you’re not makingSAMPLE SIZE – The indispensable A/B test calculation that you’re not making
SAMPLE SIZE – The indispensable A/B test calculation that you’re not making
 
[HCMC STC Jan 2015] Creating a Better Testing Future
[HCMC STC Jan 2015] Creating a Better Testing Future[HCMC STC Jan 2015] Creating a Better Testing Future
[HCMC STC Jan 2015] Creating a Better Testing Future
 
Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness
Tis The Season: Load Testing Tips and Checklist for Retail Seasonal ReadinessTis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness
Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness
 
Competency Center: retaining experts, growing expertise, maximizing income (O...
Competency Center: retaining experts, growing expertise, maximizing income (O...Competency Center: retaining experts, growing expertise, maximizing income (O...
Competency Center: retaining experts, growing expertise, maximizing income (O...
 
A/B Testing and the Infinite Monkey Theory
A/B Testing and the Infinite Monkey TheoryA/B Testing and the Infinite Monkey Theory
A/B Testing and the Infinite Monkey Theory
 
15 cases of testing with data & psychology #CRO
15 cases of testing with data & psychology #CRO15 cases of testing with data & psychology #CRO
15 cases of testing with data & psychology #CRO
 
Testing Small Agile Projects from Agile Vancouver Quality Conference 2014
Testing Small Agile Projects from Agile Vancouver Quality Conference 2014Testing Small Agile Projects from Agile Vancouver Quality Conference 2014
Testing Small Agile Projects from Agile Vancouver Quality Conference 2014
 
The Hard life of Agile Coach Project in a ruin
The Hard life of Agile Coach Project in a ruinThe Hard life of Agile Coach Project in a ruin
The Hard life of Agile Coach Project in a ruin
 
When Support Calls
When Support CallsWhen Support Calls
When Support Calls
 
[HCMC STC Jan 2015] Choosing The Best Of The Plan-Driven And Agile Developmen...
[HCMC STC Jan 2015] Choosing The Best Of The Plan-Driven And Agile Developmen...[HCMC STC Jan 2015] Choosing The Best Of The Plan-Driven And Agile Developmen...
[HCMC STC Jan 2015] Choosing The Best Of The Plan-Driven And Agile Developmen...
 
[HCMC STC Jan 2015] Developing an Offshore Context-Driven Testing Team
[HCMC STC Jan 2015] Developing an Offshore Context-Driven Testing Team[HCMC STC Jan 2015] Developing an Offshore Context-Driven Testing Team
[HCMC STC Jan 2015] Developing an Offshore Context-Driven Testing Team
 
Warshunckiy- questions to assess project maturity
Warshunckiy- questions to assess  project maturityWarshunckiy- questions to assess  project maturity
Warshunckiy- questions to assess project maturity
 
Let's Build a Product Development Organization!
Let's Build a Product Development Organization!Let's Build a Product Development Organization!
Let's Build a Product Development Organization!
 
Pavel Kamyshov "Team Health Check" Kyiv PM Club
Pavel Kamyshov "Team Health Check" Kyiv PM ClubPavel Kamyshov "Team Health Check" Kyiv PM Club
Pavel Kamyshov "Team Health Check" Kyiv PM Club
 
Kanban na lodówce
Kanban na lodówceKanban na lodówce
Kanban na lodówce
 
Value Metric - как использовать бизнес-метрики в agile-проектах
Value Metric  - как использовать бизнес-метрики в agile-проектахValue Metric  - как использовать бизнес-метрики в agile-проектах
Value Metric - как использовать бизнес-метрики в agile-проектах
 
Developing your Developers: Constructing Career Paths for your Technologists ...
Developing your Developers: Constructing Career Paths for your Technologists ...Developing your Developers: Constructing Career Paths for your Technologists ...
Developing your Developers: Constructing Career Paths for your Technologists ...
 
A/B Testing - How to Break Things and Fail Fast...without Breaking Things
A/B Testing - How to Break Things and Fail Fast...without Breaking ThingsA/B Testing - How to Break Things and Fail Fast...without Breaking Things
A/B Testing - How to Break Things and Fail Fast...without Breaking Things
 

Viewers also liked

How Spotify Does Test Automation - Kristian Karl
How Spotify Does Test Automation - Kristian KarlHow Spotify Does Test Automation - Kristian Karl
How Spotify Does Test Automation - Kristian KarlSmartBear
 
Creating Maintainable Automated Acceptance Tests
Creating Maintainable Automated Acceptance TestsCreating Maintainable Automated Acceptance Tests
Creating Maintainable Automated Acceptance TestsJez Humble
 
Quality Built In @ Spotify
Quality Built In @ SpotifyQuality Built In @ Spotify
Quality Built In @ SpotifyAndrii Dzynia
 
Introduction to SoapUI day 1
Introduction to SoapUI day 1Introduction to SoapUI day 1
Introduction to SoapUI day 1Qualitest
 
Introduction to Test Automation
Introduction to Test AutomationIntroduction to Test Automation
Introduction to Test AutomationPekka Klärck
 
Tips for Designing, Testing & Delivering eLearning in a Multi-device World
Tips for Designing, Testing & Delivering eLearning in a Multi-device WorldTips for Designing, Testing & Delivering eLearning in a Multi-device World
Tips for Designing, Testing & Delivering eLearning in a Multi-device WorldUpside Learning Solutions
 
Building Real-Time Web Applications
Building Real-Time Web ApplicationsBuilding Real-Time Web Applications
Building Real-Time Web ApplicationsTony Abou-Assaleh
 
HTML5 WebSockets in Python/Django
HTML5 WebSockets in Python/DjangoHTML5 WebSockets in Python/Django
HTML5 WebSockets in Python/DjangoTony Abou-Assaleh
 
QA процесс, часть 2
QA процесс, часть 2QA процесс, часть 2
QA процесс, часть 2DressTester
 
Office 365 Groups? Microsoft Teams? … Confused? Here's some help.
Office 365 Groups? Microsoft Teams? … Confused? Here's some help.Office 365 Groups? Microsoft Teams? … Confused? Here's some help.
Office 365 Groups? Microsoft Teams? … Confused? Here's some help.Adam Levithan
 
Software Testing Maturity Model and Assessment by Abstracta
Software Testing Maturity Model and Assessment by AbstractaSoftware Testing Maturity Model and Assessment by Abstracta
Software Testing Maturity Model and Assessment by AbstractaKalei White
 
Talking to Humans: The Art of Customer Discovery
Talking to Humans: The Art of Customer DiscoveryTalking to Humans: The Art of Customer Discovery
Talking to Humans: The Art of Customer DiscoveryNew York University
 
DevOps Patterns - Team Topologies
DevOps Patterns -  Team TopologiesDevOps Patterns -  Team Topologies
DevOps Patterns - Team TopologiesMatthew Skelton
 
500’s Demo Day Batch 11 >> CloudAcademy
500’s Demo Day Batch 11 >> CloudAcademy500’s Demo Day Batch 11 >> CloudAcademy
500’s Demo Day Batch 11 >> CloudAcademy500 Startups
 
500’s Demo Day Batch 13 >> Green Way
500’s Demo Day Batch 13 >> Green Way 500’s Demo Day Batch 13 >> Green Way
500’s Demo Day Batch 13 >> Green Way 500 Startups
 
500’s Demo Day Batch 17 >> Cladwell
500’s Demo Day Batch 17 >> Cladwell500’s Demo Day Batch 17 >> Cladwell
500’s Demo Day Batch 17 >> Cladwell500 Startups
 
500’s Demo Day Batch 15 >> SetScouter
500’s Demo Day Batch 15 >> SetScouter500’s Demo Day Batch 15 >> SetScouter
500’s Demo Day Batch 15 >> SetScouter500 Startups
 

Viewers also liked (20)

How Spotify Does Test Automation - Kristian Karl
How Spotify Does Test Automation - Kristian KarlHow Spotify Does Test Automation - Kristian Karl
How Spotify Does Test Automation - Kristian Karl
 
Creating Maintainable Automated Acceptance Tests
Creating Maintainable Automated Acceptance TestsCreating Maintainable Automated Acceptance Tests
Creating Maintainable Automated Acceptance Tests
 
Quality Built In @ Spotify
Quality Built In @ SpotifyQuality Built In @ Spotify
Quality Built In @ Spotify
 
Introduction to SoapUI day 1
Introduction to SoapUI day 1Introduction to SoapUI day 1
Introduction to SoapUI day 1
 
Introduction to Test Automation
Introduction to Test AutomationIntroduction to Test Automation
Introduction to Test Automation
 
Tips for Designing, Testing & Delivering eLearning in a Multi-device World
Tips for Designing, Testing & Delivering eLearning in a Multi-device WorldTips for Designing, Testing & Delivering eLearning in a Multi-device World
Tips for Designing, Testing & Delivering eLearning in a Multi-device World
 
When qa sucks
When qa sucksWhen qa sucks
When qa sucks
 
Building Real-Time Web Applications
Building Real-Time Web ApplicationsBuilding Real-Time Web Applications
Building Real-Time Web Applications
 
HTML5 WebSockets in Python/Django
HTML5 WebSockets in Python/DjangoHTML5 WebSockets in Python/Django
HTML5 WebSockets in Python/Django
 
QA процесс, часть 2
QA процесс, часть 2QA процесс, часть 2
QA процесс, часть 2
 
Office 365 Groups? Microsoft Teams? … Confused? Here's some help.
Office 365 Groups? Microsoft Teams? … Confused? Here's some help.Office 365 Groups? Microsoft Teams? … Confused? Here's some help.
Office 365 Groups? Microsoft Teams? … Confused? Here's some help.
 
Software Testing Maturity Model and Assessment by Abstracta
Software Testing Maturity Model and Assessment by AbstractaSoftware Testing Maturity Model and Assessment by Abstracta
Software Testing Maturity Model and Assessment by Abstracta
 
Talking to Humans: The Art of Customer Discovery
Talking to Humans: The Art of Customer DiscoveryTalking to Humans: The Art of Customer Discovery
Talking to Humans: The Art of Customer Discovery
 
DevOps Patterns - Team Topologies
DevOps Patterns -  Team TopologiesDevOps Patterns -  Team Topologies
DevOps Patterns - Team Topologies
 
VenueSpot
VenueSpotVenueSpot
VenueSpot
 
500’s Demo Day Batch 11 >> CloudAcademy
500’s Demo Day Batch 11 >> CloudAcademy500’s Demo Day Batch 11 >> CloudAcademy
500’s Demo Day Batch 11 >> CloudAcademy
 
500’s Demo Day Batch 13 >> Green Way
500’s Demo Day Batch 13 >> Green Way 500’s Demo Day Batch 13 >> Green Way
500’s Demo Day Batch 13 >> Green Way
 
AppGrooves
AppGrooves AppGrooves
AppGrooves
 
500’s Demo Day Batch 17 >> Cladwell
500’s Demo Day Batch 17 >> Cladwell500’s Demo Day Batch 17 >> Cladwell
500’s Demo Day Batch 17 >> Cladwell
 
500’s Demo Day Batch 15 >> SetScouter
500’s Demo Day Batch 15 >> SetScouter500’s Demo Day Batch 15 >> SetScouter
500’s Demo Day Batch 15 >> SetScouter
 

Similar to Ross Snyder, Etsy, SXSW Lean Startup 2013

Continuous Deployment at Etsy: A Tale of Two Approaches
Continuous Deployment at Etsy: A Tale of Two ApproachesContinuous Deployment at Etsy: A Tale of Two Approaches
Continuous Deployment at Etsy: A Tale of Two ApproachesRoss Snyder
 
The Business Case for DevOps - Justifying the Journey
The Business Case for DevOps - Justifying the JourneyThe Business Case for DevOps - Justifying the Journey
The Business Case for DevOps - Justifying the JourneyXebiaLabs
 
Life in the tech trenches (2015)
Life in the tech trenches (2015)Life in the tech trenches (2015)
Life in the tech trenches (2015)Julien SIMON
 
CTO Crunch avec Julien Simon, Viadeo
CTO Crunch avec Julien Simon, ViadeoCTO Crunch avec Julien Simon, Viadeo
CTO Crunch avec Julien Simon, ViadeoFrance Digitale
 
Practical agile TechExeter
Practical agile TechExeterPractical agile TechExeter
Practical agile TechExeterIan Ames
 
Practical Agile. Lessons learned the hard way on our journey building digita...
Practical Agile.  Lessons learned the hard way on our journey building digita...Practical Agile.  Lessons learned the hard way on our journey building digita...
Practical Agile. Lessons learned the hard way on our journey building digita...TechExeter
 
How To (Not) Open Source - Javazone, Oslo 2014
How To (Not) Open Source - Javazone, Oslo 2014How To (Not) Open Source - Javazone, Oslo 2014
How To (Not) Open Source - Javazone, Oslo 2014gdusbabek
 
Continuous Deployment
Continuous DeploymentContinuous Deployment
Continuous DeploymentBrian Henerey
 
#Measurecamp : 18 Simple Ways to F*** up Your AB Testing
#Measurecamp : 18 Simple Ways to F*** up Your AB Testing#Measurecamp : 18 Simple Ways to F*** up Your AB Testing
#Measurecamp : 18 Simple Ways to F*** up Your AB TestingCraig Sullivan
 
6 ways DevOps helped PrepSportswear move from monolith to microservices
6 ways DevOps helped PrepSportswear move from monolith to microservices6 ways DevOps helped PrepSportswear move from monolith to microservices
6 ways DevOps helped PrepSportswear move from monolith to microservicesDynatrace
 
Agile Development Overview (with a bit about builds)
Agile Development Overview (with a bit about builds)Agile Development Overview (with a bit about builds)
Agile Development Overview (with a bit about builds)David Benjamin
 
2012 SxSW When IT Says No by Gene Kim
2012 SxSW When IT Says No by Gene Kim2012 SxSW When IT Says No by Gene Kim
2012 SxSW When IT Says No by Gene KimGene Kim
 
Ericriesleanstartuppresentationforweb2
Ericriesleanstartuppresentationforweb2Ericriesleanstartuppresentationforweb2
Ericriesleanstartuppresentationforweb2Edmund FOng
 
Problem Solving Frameworks
Problem Solving FrameworksProblem Solving Frameworks
Problem Solving FrameworksAlan McCloy
 
Building Startups and Minimum Viable Products (NDC2013)
Building Startups and Minimum Viable Products (NDC2013)Building Startups and Minimum Viable Products (NDC2013)
Building Startups and Minimum Viable Products (NDC2013)Ben Hall
 
(SPOT205) 5 Lessons for Managing Massive IT Transformation Projects
(SPOT205) 5 Lessons for Managing Massive IT Transformation Projects(SPOT205) 5 Lessons for Managing Massive IT Transformation Projects
(SPOT205) 5 Lessons for Managing Massive IT Transformation ProjectsAmazon Web Services
 
Bigger Unit Test Are Better
Bigger Unit Test Are BetterBigger Unit Test Are Better
Bigger Unit Test Are BetterPeter Schuler
 

Similar to Ross Snyder, Etsy, SXSW Lean Startup 2013 (20)

Continuous Deployment at Etsy: A Tale of Two Approaches
Continuous Deployment at Etsy: A Tale of Two ApproachesContinuous Deployment at Etsy: A Tale of Two Approaches
Continuous Deployment at Etsy: A Tale of Two Approaches
 
The Business Case for DevOps - Justifying the Journey
The Business Case for DevOps - Justifying the JourneyThe Business Case for DevOps - Justifying the Journey
The Business Case for DevOps - Justifying the Journey
 
Life in the tech trenches (2015)
Life in the tech trenches (2015)Life in the tech trenches (2015)
Life in the tech trenches (2015)
 
CTO Crunch avec Julien Simon, Viadeo
CTO Crunch avec Julien Simon, ViadeoCTO Crunch avec Julien Simon, Viadeo
CTO Crunch avec Julien Simon, Viadeo
 
Practical agile TechExeter
Practical agile TechExeterPractical agile TechExeter
Practical agile TechExeter
 
Practical Agile. Lessons learned the hard way on our journey building digita...
Practical Agile.  Lessons learned the hard way on our journey building digita...Practical Agile.  Lessons learned the hard way on our journey building digita...
Practical Agile. Lessons learned the hard way on our journey building digita...
 
How To (Not) Open Source - Javazone, Oslo 2014
How To (Not) Open Source - Javazone, Oslo 2014How To (Not) Open Source - Javazone, Oslo 2014
How To (Not) Open Source - Javazone, Oslo 2014
 
AgileCamp 2014 Track 5: The Seven Wastes - Can You Get Leaner
AgileCamp 2014 Track 5: The Seven Wastes - Can You Get LeanerAgileCamp 2014 Track 5: The Seven Wastes - Can You Get Leaner
AgileCamp 2014 Track 5: The Seven Wastes - Can You Get Leaner
 
Continuous Deployment
Continuous DeploymentContinuous Deployment
Continuous Deployment
 
#Measurecamp : 18 Simple Ways to F*** up Your AB Testing
#Measurecamp : 18 Simple Ways to F*** up Your AB Testing#Measurecamp : 18 Simple Ways to F*** up Your AB Testing
#Measurecamp : 18 Simple Ways to F*** up Your AB Testing
 
6 ways DevOps helped PrepSportswear move from monolith to microservices
6 ways DevOps helped PrepSportswear move from monolith to microservices6 ways DevOps helped PrepSportswear move from monolith to microservices
6 ways DevOps helped PrepSportswear move from monolith to microservices
 
Agile Development Overview (with a bit about builds)
Agile Development Overview (with a bit about builds)Agile Development Overview (with a bit about builds)
Agile Development Overview (with a bit about builds)
 
Great! another bug
Great! another bugGreat! another bug
Great! another bug
 
2012 SxSW When IT Says No by Gene Kim
2012 SxSW When IT Says No by Gene Kim2012 SxSW When IT Says No by Gene Kim
2012 SxSW When IT Says No by Gene Kim
 
Ericriesleanstartuppresentationforweb2
Ericriesleanstartuppresentationforweb2Ericriesleanstartuppresentationforweb2
Ericriesleanstartuppresentationforweb2
 
Problem Solving Frameworks
Problem Solving FrameworksProblem Solving Frameworks
Problem Solving Frameworks
 
No Estimates - Hai Dang
No Estimates - Hai DangNo Estimates - Hai Dang
No Estimates - Hai Dang
 
Building Startups and Minimum Viable Products (NDC2013)
Building Startups and Minimum Viable Products (NDC2013)Building Startups and Minimum Viable Products (NDC2013)
Building Startups and Minimum Viable Products (NDC2013)
 
(SPOT205) 5 Lessons for Managing Massive IT Transformation Projects
(SPOT205) 5 Lessons for Managing Massive IT Transformation Projects(SPOT205) 5 Lessons for Managing Massive IT Transformation Projects
(SPOT205) 5 Lessons for Managing Massive IT Transformation Projects
 
Bigger Unit Test Are Better
Bigger Unit Test Are BetterBigger Unit Test Are Better
Bigger Unit Test Are Better
 

More from 500 Startups (20)

Get on Board
Get on BoardGet on Board
Get on Board
 
Connected Analytics
Connected AnalyticsConnected Analytics
Connected Analytics
 
Sira Medical
Sira MedicalSira Medical
Sira Medical
 
The Atlas
The AtlasThe Atlas
The Atlas
 
Trash Warrior
Trash WarriorTrash Warrior
Trash Warrior
 
Thematic
ThematicThematic
Thematic
 
Shiplyst
ShiplystShiplyst
Shiplyst
 
Renetec
RenetecRenetec
Renetec
 
Predina
PredinaPredina
Predina
 
Pluto
PlutoPluto
Pluto
 
Plant an App
Plant an AppPlant an App
Plant an App
 
Pilota
PilotaPilota
Pilota
 
Mero Technologies
Mero TechnologiesMero Technologies
Mero Technologies
 
Omnitron Sensors
Omnitron SensorsOmnitron Sensors
Omnitron Sensors
 
Juked
JukedJuked
Juked
 
GamerzClass
GamerzClassGamerzClass
GamerzClass
 
eino
einoeino
eino
 
Cenos
CenosCenos
Cenos
 
Bliinx
BliinxBliinx
Bliinx
 
Butlr
ButlrButlr
Butlr
 

Ross Snyder, Etsy, SXSW Lean Startup 2013

  • 1. Continuous Deployment at : A Tale of Two Approaches Ross Snyder ross@etsy.com @beamrider9 March 9, 2013
  • 3. is: The global marketplace we make together.
  • 4. is: The premier destination for handmade goods, vintage items, and craft supplies. simplertimestoys lacklusterco norwesterseaglass
  • 5. quick facts: (as of March 2013) • 22+ million members • 800,000+ active shops • 18+ million items currently for sale • 20 cents to list item, 3.5% transaction fee • 400+ employees (majority in Brooklyn)
  • 6. Since opening its doors in June 2005, Etsy has grown virtually non-stop. $1,000 $800 $600 $400 $200 $0 2005 2006 2007 2008 2009 2010 2011 2012 Gross Merchandise Sales ($MM)
  • 7. A nice problem to have: “Our site is so successful, how can we move fast enough to keep up with demand?”
  • 9. : The Early Years (2005 - 2008)
  • 10. : The Early Years 1. Spend significant time writing code
  • 11. : The Early Years 1. Weeks writing code 2. Painful source control merge
  • 12. : The Early Years 1. Weeks writing code 2. Painful merge 3. Hand off to someone else to deploy
  • 13. : The Early Years 1. Weeks writing code 2. Painful merge 3. Hand off to deployers 4. Deploy, site goes down
  • 14. : The Early Years 1. Weeks writing code 4. Deploy, site down 2. Painful merge 3. Hand off to deployers 5. Roll back deploy
  • 15. : The Early Years 1. Weeks writing code 4. Deploy, site down 2. Painful merge 5. Roll back deploy 3. Hand off to deployers 6. Spend hours (days?) fixing bugs
  • 16. : The Early Years 1. Weeks writing code 4. Deploy, site down 2. Painful merge 5. Roll back deploy 3. Hand off to deployers 6. Fix bugs 7. Go back to step 2
  • 17. : The Early Years WATERFALL!
  • 18. : The Early Years Pros: Early Etsy engineers used this release cycle to bootstrap the marketplace from nothing. Forever grateful.
  • 19. : The Early Years Cons: • Large changesets • Infrequent deploys • Weak confidence in deploy success • Significant time spent deploying • Low ability to experiment/iterate/react • Developer stress/unhappiness
  • 20. : The Early Years By late 2008, Etsy is still a startup, but has the deploy process of a much bulkier company. Popularity is on the verge of outpacing capacity.
  • 22. : Today 1. Small changesets, deployed frequently
  • 23. : Today 1. Small changesets 2. Engineers deploy the site
  • 24. : Today And not just engineers, but also: • Designers • Product Folks • Upper Management • Board Members • Dogs
  • 25. : Today 1. Small changesets 2. Engineers deploy 3. Deploys are fast and near-effortless
  • 26. : Today 1. Small changesets 2. Engineers deploy 3. Deploys are fast 4. Most changes behind config flags (safer deploys)
  • 27. : Today 1. Small changesets 4. Changes behind flags 2. Engineers deploy 3. Deploys are fast 5. Graphs/metrics to assess deploy
  • 28. : Today 1. Small changesets 4. Changes behind flags 2. Engineers deploy 5. Copious graphs/metrics 3. Deploys are fast 6. If issues, fix immediately & roll forward
  • 29. : Today This isn’t license to break stuff, quickly. Engineer-driven QA and solid unit testing are integral parts of the process.
  • 30. : Today 1. Small changesets 4. Changes behind flags 2. Engineers deploy 5. Copious graphs/metrics 3. Deploys are fast 6. Fix fast & roll forward 7. Repeat 25+ times per day, every day
  • 31. Then: 1. Weeks writing code 2. Painful merge 3. Hand off to deployers 4. Deploy, site down 5. Roll back deploy 6. Fix bugs, go to step 2 Now: 1. Small changesets 2. Engineers deploy 3. Deploys are fast 4. Changes behind flags 5. Copious graphs/metrics 6. Fix fast & roll forward
  • 32. Etsy Deploy Stats: 2012 • Deployed to production 6,419 times • On average, 535/month, 25/day • Additional 3,851 config-only deploys • 196 different people deployed to prod • Nov/Dec 2012: deployed 752 times
  • 33. Why does it work?
  • 34. Continuous Deployment Math • N = # of deploys • P = probability of site degradation • S = average severity of degradation • T = time to detect/resolve Expected = N*P*S*T Downtime
  • 35. Continuous Deployment Math N = # of deploys S = avg. severity of degradation P = prob. of degradation T = time to detect/resolve Before: Now: •N=1 • N = 250 ↑↑↑↑ • P = 0.5 • P = 0.1 ↓ • S = 0.7 • S = 0.05 ↓↓ • T = 100 • T = 5 ↓↓↓ E.D. = 35 E.D. = 6.25 (all numbers completely arbitrary)
  • 36. Big Takeaway Etsy circa 2013 (400+ employees) acts, in some ways, more like a startup than Etsy circa 2008 (40+ employees).
  • 37. Continuous Deployment makes possible: “Continuous Experimentation” http://etsy.me/continuous-experimentation
  • 38. Continuous Experimentation 1. Small changes 2. Run experiment (A/B test) 3. Analyze data 4. Re-examine assumptions Repeat continuously in pursuit of larger goals.
  • 39. Heard since 2010: “Neat experiment, but this will never scale.” As of 2013, Etsy has 100+ engineers - still going strong.
  • 40. Some Etsy Customizations Deploying is a first-class feature. Inability to deploy is a P1 incident (same as site down).
  • 41. Some Etsy Customizations We continuously deploy not just the main Etsy website, but as much as possible: • Internal admin site • API • Big data • Search • Blog • Deployinator itself
  • 42. Some Etsy Customizations In the rare case we can’t continuously deploy, we create alternative tools: • Database schema changes • PCI-DSS environment (credit cards) We do continuously deploy as much of our payment processing as is safe & legal (98%).
  • 43. Some Etsy Customizations Keeping deploys fast is paramount and worth the investment in manpower & hardware.
  • 44. Some Etsy Customizations Continuous deployment is all about moving forward, sometimes at the expense of the past. Our solution: engineering-wide bug rotation, one day a month, every engineer participates.
  • 45. Fun Fact: Continuous Deployment is a fantastic recruitment tool for attracting engineers who like to move fast and get stuff done.
  • 46. Learn more: http://codeascraft.etsy.com/ Etsy open source (Deployinator, StatsD) http://etsy.github.com/ Join the fun: http://www.etsy.com/careers