0
Principles and Practices in
Continuous Deployment
Mike Brittain
Engineering Director, Etsy
@mikebrittain mikebrittain.com/...
“Continuous Deployment”
Process by which our team deploys software changes
to production services over 30 times per day.
Where we started
Principles for our Engineering team
Continuous Deployment
Business case
Five years ago…
2-3 weeks of code changes
Release and rollback plans
Traffic and infrastructure management (Ops)
6-14 hours
Five years ago…
“Deployment Army”
Stressful, especially when things go wrong
Long days and late nights
Scheduled downtime
pro·duc·tion [pruh-duhk-shuhn] (n)
1. This complex system of application code,
distributed services, servers, networking g...
Software for large-scale web sites has been
traditionally written by one group of people, then
released and operated by a ...
Stagnation
“…frequent and prolonged outages.”
2010 CAPACITY PLAN
First, Principles.
Innovate or die
Innovate or die
Resolve scaling hurdles
Innovate or die
Resolve scaling hurdles
Mean-time-to-recovery
“Quality is not just testing pre-release.
It also includes our adaptability and
response time.”
- Jeff Sussna at ALM Forum...
Innovate or die
Resolve scaling hurdles
Mean-time-to-recovery
Innovate or die
Resolve scaling hurdles
Mean-time-to-recovery
Healthy and talented engineering team
Autonomy, Mastery, Purpose.
“Drive: The surprising truth about what
motivates us.” ~Dan Pink, at RSA
http://youtu.be/u6XAP...
Innovate or die
Resolve scaling hurdles
Mean-time-to-recovery
Healthy and talented engineering team
Stop stressing about r...
First, Principles.
http://timothyfitz.com/2009/02/10/continuous-deployment-at-imvu-doing-the-impossible-fifty-times-a-day/
In a software rele...
http://youtu.be/LdOe18KhtT4
SECRET WEAPON:

Hired as VP, Tech-Ops at Etsy
Continuous Deployment
Continuous Delivery
~ vs ~
Frequent check-ins directly to
mainline.
Continuous Deployment Continuous Delivery
✓ ✓
Continuous Integration and
Automated tests.
Continuous Deployment Continuous Delivery
✓ ✓
Keep the build green.
We’re always ready to release.
Continuous Deployment Continuous Delivery
✓ ✓
“One button” deploys.
Continuous Deployment Continuous Delivery
✓ ✓
Business dictates when a build is
deployed.
Continuous Deployment Continuous Delivery
✓
Every passing build is deployed to
production.
Continuous Deployment Continuous Delivery
✓
All enhancements are gated by
Config Flags. (“Branch in code”)
Continuous Deployment Continuous Delivery
✓ ?
Most of the builds we deploy are
“dark” changes.
CSS rules and properties
Copy in templates (e.g. typos)
New, un-referenced code (e.g. classes, funcs, templates)
Code path...
Dev Team Version Control
Build & Unit
Tests
Automated
Acceptance Tests
User Acceptance
Tests
Release
Check in
Trigger
Feed...
Dev Team Version Control
Build & Unit
Tests
Automated
Acceptance Tests
User Acceptance
Tests
Release
Check in
Trigger
Feed...
Dev Team Version Control
Build & Unit
Tests
Automated
Acceptance Tests
User Acceptance
Tests
Release
Check in
Trigger
Feed...
Dev Team Version Control
Build & Unit
Tests
Automated
Acceptance Tests
User Acceptance
Tests
Release
Check in
Trigger
Feed...
Dev Team Version Control
Build & Unit
Tests
Automated
Acceptance Tests
User Acceptance
Tests
Release
Check in
Trigger
Feed...
“What do you mean, ‘it’s not working in
production?’ I TESTED IT BEFORE WE
RELEASED!”
Dev Team Version Control
Build & Unit
Tests
Automated
Acceptance Tests
User Acceptance
Tests
Release
Check in
Trigger
Trig...
Dev Team Version Control
Build & Unit
Tests
Automated
Acceptance Tests
User Acceptance
Tests
Release
Check in
Trigger
Trig...
Dev Team Version Control
Build & Unit
Tests
Automated
Acceptance Tests
User Acceptance
Tests
Release
Check in
Trigger
Trig...
Dev Team Version Control
Build & Unit
Tests
Automated
Acceptance Tests
User Acceptance
Tests
Release
Check in
Trigger
Trig...
Dev Team Version Control
Build & Unit
Tests
Automated
Acceptance Tests
User Acceptance
Tests
Release
Check in
Trigger
Trig...
Dev Team Version Control
Build & Unit
Tests
Automated
Acceptance Tests
User Acceptance
Tests
Release
Check in
Trigger
Trig...
We aim to reduce fundamental surprise in
every release.
Furthermore, we optimize for detecting
and recovering from failures quickly.
Pre-production validation
Code deployed to de-pooled application (web) servers
touching prod services and databases.
Smoke...
Production validation
Exactly the same server configs, services and data as
pre-prod, but this is where we introduce appli...
Production validation
Exactly the same server configs, services and data as
pre-prod, but this is where we introduce appli...
Single
release
Many
releases
50K LOC/month
Few opportunities for failure

Wide surface area (50,000 LOC)
High MTTR
!
All o...
Monitoring
Monitoring
PHP Warnings Bug Reports and Help Requests
Deploy logs
Post-Mortems
Check in
Trigger
Feedback Trigger
Feedback Approval
Approval
Smoke Tests
User Acceptance!
Tests
ReleaseDev Team Version Co...
Dev Pre-Production
(“Princess”)
Production
Check in
Trigger
Feedback Trigger
Feedback Approval
ApprovalFeedback
Smoke Test...
Dev Pre-Production
(“Princess”)
Production
Check in
Trigger
Feedback Trigger
Feedback Approval
ApprovalFeedback
Smoke Test...
Dev Pre-Production
(“Princess”)
Production
Check in
Trigger
Feedback Trigger
Feedback Approval
ApprovalFeedback
Smoke Test...
“Allow buttons properly to inherit color
from their parent node.”
Five years ago…
2-3 weeks of code changes
Release and rollback plans
Traffic and infrastructure management (Ops)
6-14 hours
Five years ago…
“Deployment Army”
Stressful, especially when things go wrong
Long days and late nights
Scheduled downtime
Why do we do this?
Innovate or die.
Resolve scaling hurdles.
Mean-time-to-recovery.
Healthy and talented engineering team.
Stop stressing abo...
Innovate or die.
Resolve scaling hurdles.
Mean-time-to-recovery.
Healthy and talented engineering team.
Stop stressing abo...
Admin-launch and whitelist
Ramp-up public traffic
mainline
header_redesign
search_filter_custom_orders
checkout_blue_button
listing_css_refactor
www.etsy.com
beta01.etsy.com
beta02.etsy.com
beta03.etsy.com
www.etsy.com
beta01.etsy.com
beta02.etsy.com
beta03.etsy.com
US","region":"US","detected_currency_code":"USD","detected_language":"en-
US","detected_region":"US","accept-languages":"e...
Catapult
Observed impact
Time series data for
duration of the
experiment
Observed impact
Time series data for
duration of the
experiment
Frank
Product Manager
“I want to find out whether
buyers will favor a single
price for the product that
includes shipping.”
https://www.etsy.com...
Eligibility requirements:
- Must be first page of visit
- Buyer & seller in same region
- etc…
Time: < 8 hours
Staff: One
!
Design, config flag (disabled), eligibility code in
controller, template code, CSS, code revi...
We do not bundle the item
price and shipping cost
together today.
!
https://www.etsy.com/shop/lucra
Ambitious Product Goal
Ambitious Product Goal
Monolithic
Building and measuring many
things at once.
Ambitious Product Goal
Monolithic
Building and measuring many
things at once.
Iterative
One thing at a time, our design
go...
Time: < 8 hours
Staff: One
!
Design, config flag (disabled), eligibility code in
controller, template code, CSS, code revi...
Is this for me?
http://timothyfitz.com/2009/02/10/continuous-deployment-at-imvu-doing-the-impossible-fifty-times-a-day/
“Maybe this is jus...
etsystatus.com
@mikebrittain
Very end of 2009 Today
DEPLOYMENTSPERDAY
APPCODE
CONFIGFILES
$1.35 Billion Goods sold in 2013
60+ Million Unique visitors per month
!
175+ Committers, everyone deploys
http://www.etsy...
Thank you.
Mike Brittain
Engineering Director, Etsy
@mikebrittain mikebrittain.com/talks
Principles and Practices in Continuous Deployment at Etsy
Principles and Practices in Continuous Deployment at Etsy
Principles and Practices in Continuous Deployment at Etsy
Principles and Practices in Continuous Deployment at Etsy
Principles and Practices in Continuous Deployment at Etsy
Principles and Practices in Continuous Deployment at Etsy
Upcoming SlideShare
Loading in...5
×

Principles and Practices in Continuous Deployment at Etsy

13,519

Published on

Presented at ALM Forum 2014.

Like what you've read? We're frequently hiring for a variety of engineering roles at Etsy. If you're interested, drop me a line or send me your resume: mike@etsy.com.

http://www.etsy.com/careers

Published in: Engineering, Internet, Technology
4 Comments
95 Likes
Statistics
Notes
  • Great presentation ... I was wondering if Etsy does dogfooding, and A/B Testing ... does Etsy perform any kind of packaging mechanism to deploy the executable ?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • my c­­­­o-­­­­wor­­­­ker's ha­­­­lf-si­­­­ster ma­­­­kes $8­­­­9 /hou­­­­r o­­­­n t­­­­he l­­­­aptop . Sh­­­­e ha­­­­s be­­­­en fire­­­­d fro­­­­m w­­­­ork f­­­­or five mo­­­­nths ­­­­b­­­­­­­­ut las­­­­t m­­­­onth h­­­­er c­­­­heck wa­­­­s $21­­­­382 ju­­­­st ­­­­wor­­­­ki­­­­ng o­­­­n th­­­­e la­­­­ptop fo­­­­r a f­­­­ew h­­­­ours.

    l­­­­ook a­­­­t th­­­­is s­­­­ite........ http://goo.gl/ZC87k7
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • ddffhhj
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • ALL FAILED PROBLEMS BY OTHER HEALERS CAN BE SOLVED BY MAAMA SHAHIEDA AT A LOW COST WITHOUT FAIL! ITS 100% GUARANTEED!

    28 years + full time experience & over 4800 satisfied clients World Wide

    Are Any Of The Serious Problems Listed Below (Or Similar) Adversely Affecting You Or Your Loved Ones...And You Are Feeling 'Powerless' To Find A Solution?

    *I BRING BACK LOST LOVERS IN 24hrs.

    *REMOTE CONTROL OVER LOVERS.

    *ALL TYPES OF SPELL CASTING & VOODOO WORKER.

    * IS YOUR HEALER TAKING LONG TO SOLVE YOUR PROBLEMS? THIS IS YOUR CHANCE TO BE HELPED QUICKLY.

    *GET RIDE OF ANY WITCHCRAFT, BAD LUCK & CURSES.

    * EXPERT IN DISTANCE HEALING.

    *COURT CASES EVEN IF ONE IS CONVICTED[IF THERE'S A CHANCE OF APPEALING]

    *IS SOME BODY JEALOUS WITH YOUR RELATIONSHIP?

    *UNFINISHED JOBS BY OTHER DOCTORS-IF NOT SATISFIED-COME TO ME.

    *ARE YOU BEING FORCED INTO A DIVORCE?

    *I PREVENT BAD LUCK AND JEALOUSY.

    *HEALERS WHO NEEDS MORE POWERS TO HEAL THEIR

    *I CAN STOP SOME ONE INTERFERING WITH YOUR RELATIONSHIP.

    * DO YOU WANT TO HAVE CHILDREN OF YOUR CHOICE?

    *DO YOU WANT QUICK MARRIAGE PROPOSAL?

    *I CAN CLEAN PEOPLE, HOUSES,BUSINESSES AND PROPERTIES FROM BAD LUCK.

    *HIRE SHORT BOYS TO GIVE YOU RICHNESS IN 1 WEEK.$3800

    *BUY AMAYEMBE [ANCESTRAL SPIRITS] TO MAKE YOU RICH.$4500

    * WOMEN, COME FOR THE TAMPON. IT WILL HELP YOU NOT TO LOSE YOUR RELATION SHIPS BECAUSE IT MAKES YOU SWEET, WARM AND TIGHT.

    AND MANY MORE.....CALL: +27781419372 OR VISIT: http://www.maamashahieda.webs.com

    ARE YOU DISAPPOINTED BY OTHER DOCTORS?

    JUST TRY ME THE LEADER OF AFRICAN TRADITIONAL HERBALIST HEALER AND WITNESS THE RESULTS!

    EMAIL: maamashahieda@yahoo.com AND YOU WILL NOT REGRET! SHE'S LOCATED AT CORNER OF KLIPFONTEIN AND RUTH ROAD,RYLANDS ESTATE{CAPETOWN SOUTH AFRICA}

    'If any of the issues above resonate with you and you have nowhere else to turn I can help you no matter where you are - anywhere in the World. All work is guaranteed to be 100% remote, and 100% discreet.

    I have well over 28 years full time professional international experience, successfully healing the problems above (and more) for people like you... when absolutely nothing else would work.'
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
13,519
On Slideshare
0
From Embeds
0
Number of Embeds
19
Actions
Shares
0
Downloads
299
Comments
4
Likes
95
Embeds 0
No embeds

No notes for slide

Transcript of "Principles and Practices in Continuous Deployment at Etsy"

  1. 1. Principles and Practices in Continuous Deployment Mike Brittain Engineering Director, Etsy @mikebrittain mikebrittain.com/talks
  2. 2. “Continuous Deployment” Process by which our team deploys software changes to production services over 30 times per day.
  3. 3. Where we started Principles for our Engineering team Continuous Deployment Business case
  4. 4. Five years ago… 2-3 weeks of code changes Release and rollback plans Traffic and infrastructure management (Ops) 6-14 hours
  5. 5. Five years ago… “Deployment Army” Stressful, especially when things go wrong Long days and late nights Scheduled downtime
  6. 6. pro·duc·tion [pruh-duhk-shuhn] (n) 1. This complex system of application code, distributed services, servers, networking gear, etc., upon which we’re going to try to carefully apply a complicated set of changes and hope that nothing goes wrong. Cross your fingers… here goes.
  7. 7. Software for large-scale web sites has been traditionally written by one group of people, then released and operated by a different group. These two groups have very different levels of visibility into how the software works.
  8. 8. Stagnation
  9. 9. “…frequent and prolonged outages.” 2010 CAPACITY PLAN
  10. 10. First, Principles.
  11. 11. Innovate or die
  12. 12. Innovate or die Resolve scaling hurdles
  13. 13. Innovate or die Resolve scaling hurdles Mean-time-to-recovery
  14. 14. “Quality is not just testing pre-release. It also includes our adaptability and response time.” - Jeff Sussna at ALM Forum, 2014
  15. 15. Innovate or die Resolve scaling hurdles Mean-time-to-recovery
  16. 16. Innovate or die Resolve scaling hurdles Mean-time-to-recovery Healthy and talented engineering team
  17. 17. Autonomy, Mastery, Purpose. “Drive: The surprising truth about what motivates us.” ~Dan Pink, at RSA http://youtu.be/u6XAPnuFjJc
  18. 18. Innovate or die Resolve scaling hurdles Mean-time-to-recovery Healthy and talented engineering team Stop stressing about releases
  19. 19. First, Principles.
  20. 20. http://timothyfitz.com/2009/02/10/continuous-deployment-at-imvu-doing-the-impossible-fifty-times-a-day/ In a software release process Fail Fast means releasing undeployed code as fast as possible, instead of waiting for a weekly release to break.
  21. 21. http://youtu.be/LdOe18KhtT4
  22. 22. SECRET WEAPON:
 Hired as VP, Tech-Ops at Etsy
  23. 23. Continuous Deployment Continuous Delivery ~ vs ~
  24. 24. Frequent check-ins directly to mainline. Continuous Deployment Continuous Delivery ✓ ✓
  25. 25. Continuous Integration and Automated tests. Continuous Deployment Continuous Delivery ✓ ✓
  26. 26. Keep the build green. We’re always ready to release. Continuous Deployment Continuous Delivery ✓ ✓
  27. 27. “One button” deploys. Continuous Deployment Continuous Delivery ✓ ✓
  28. 28. Business dictates when a build is deployed. Continuous Deployment Continuous Delivery ✓
  29. 29. Every passing build is deployed to production. Continuous Deployment Continuous Delivery ✓
  30. 30. All enhancements are gated by Config Flags. (“Branch in code”) Continuous Deployment Continuous Delivery ✓ ?
  31. 31. Most of the builds we deploy are “dark” changes.
  32. 32. CSS rules and properties Copy in templates (e.g. typos) New, un-referenced code (e.g. classes, funcs, templates) Code paths behind disabled config flags etc…
  33. 33. Dev Team Version Control Build & Unit Tests Automated Acceptance Tests User Acceptance Tests Release Check in Trigger Feedback Source: http://en.wikipedia.org/wiki/Continuous_delivery Continuous Delivery release pipeline
  34. 34. Dev Team Version Control Build & Unit Tests Automated Acceptance Tests User Acceptance Tests Release Check in Trigger Feedback Check in Trigger Feedback Trigger Feedback Source: http://en.wikipedia.org/wiki/Continuous_delivery Continuous Delivery release pipeline
  35. 35. Dev Team Version Control Build & Unit Tests Automated Acceptance Tests User Acceptance Tests Release Check in Trigger Feedback Check in Trigger Feedback Trigger Feedback Check in Trigger Feedback Trigger Feedback Approval ApprovalFeedback Source: http://en.wikipedia.org/wiki/Continuous_delivery Continuous Delivery release pipeline
  36. 36. Dev Team Version Control Build & Unit Tests Automated Acceptance Tests User Acceptance Tests Release Check in Trigger Feedback Check in Trigger Feedback Trigger Feedback Check in Trigger Feedback Trigger Feedback Approval ApprovalFeedback Continuous Delivery release pipeline Dev / Integration Staging Production
  37. 37. Dev Team Version Control Build & Unit Tests Automated Acceptance Tests User Acceptance Tests Release Check in Trigger Feedback Check in Trigger Feedback Trigger Feedback Check in Trigger Feedback Trigger Feedback Approval ApprovalFeedback Continuous Delivery release pipeline Dev / Integration Staging Production Assumptions: Staging is a perfect reflection of Production, with respect to hardware, configurations, data, overall load, capacity, etc. Deploy process is infallible.
  38. 38. “What do you mean, ‘it’s not working in production?’ I TESTED IT BEFORE WE RELEASED!”
  39. 39. Dev Team Version Control Build & Unit Tests Automated Acceptance Tests User Acceptance Tests Release Check in Trigger Trigger Approval Continuous Delivery release pipeline Dev / Integration Staging Production Check in Trigger Trigger Approval Check in Trigger Trigger Approval Check in Trigger Trigger Approval Check in Trigger Trigger Approval Check in Trigger Trigger Approval
  40. 40. Dev Team Version Control Build & Unit Tests Automated Acceptance Tests User Acceptance Tests Release Check in Trigger Trigger Approval Continuous Delivery release pipeline Dev / Integration Staging Production Check in Trigger Trigger Approval Check in Trigger Trigger Approval Check in Trigger Trigger Approval Check in Trigger Trigger Approval Approval Check in Trigger Trigger Approval Feedback
  41. 41. Dev Team Version Control Build & Unit Tests Automated Acceptance Tests User Acceptance Tests Release Check in Trigger Trigger Approval Continuous Delivery release pipeline Check in Trigger Trigger Approval Check in Trigger Trigger Approval Check in Trigger Trigger Approval Check in Trigger Trigger Approval Approval Check in Trigger Trigger Approval Feedback "Because you’re integrating so frequently, there is significantly less back- tracking to discover where things went wrong , so you can spend more time building features.” ! —ThoughtWorks ! ! http://www.thoughtworks.com/continuous-integration Dev / Integration Staging Production
  42. 42. Dev Team Version Control Build & Unit Tests Automated Acceptance Tests User Acceptance Tests Release Check in Trigger Trigger Approval Continuous Delivery release pipeline Check in Trigger Trigger Approval Check in Trigger Trigger Approval Check in Trigger Trigger Approval Check in Trigger Trigger Approval Approval Check in Trigger Trigger Approval Feedback Dev / Integration Staging Production Where’s the bug? ! In one of the numerous check-ins? Missing unit tests? Missing automated UA tests? Missing manual UA tests?
  43. 43. Dev Team Version Control Build & Unit Tests Automated Acceptance Tests User Acceptance Tests Release Check in Trigger Trigger Approval Continuous Delivery release pipeline Check in Trigger Trigger Approval Check in Trigger Trigger Approval Check in Trigger Trigger Approval Check in Trigger Trigger Approval Approval Check in Trigger Trigger Approval Feedback Dev / Integration Staging Production Where’s the bug? ! In one of the numerous check-ins? Missing unit tests? Missing automated UA tests? Missing manual UA tests? ! Data out of sync? Server configurations out of sync? Capacity vs. current load? Deployment script?
  44. 44. Dev Team Version Control Build & Unit Tests Automated Acceptance Tests User Acceptance Tests Release Check in Trigger Trigger Approval Continuous Delivery release pipeline Check in Trigger Trigger Approval Check in Trigger Trigger Approval Check in Trigger Trigger Approval Check in Trigger Trigger Approval Approval Check in Trigger Trigger Approval Feedback Dev / Integration Staging Production How will we know when something is wrong in production? ! How long will it take to resolve the issue? Check in Trigger
  45. 45. We aim to reduce fundamental surprise in every release.
  46. 46. Furthermore, we optimize for detecting and recovering from failures quickly.
  47. 47. Pre-production validation Code deployed to de-pooled application (web) servers touching prod services and databases. Smoke tests Integration tests Functional tests User-Acceptance (ad hoc)
  48. 48. Production validation Exactly the same server configs, services and data as pre-prod, but this is where we introduce application code to live traffic.
  49. 49. Production validation Exactly the same server configs, services and data as pre-prod, but this is where we introduce application code to live traffic. Smoke tests (esp. over public hostnames) User-Acceptance testing behind config flags Gratuitous monitoring Customer support and forums
  50. 50. Single release Many releases 50K LOC/month Few opportunities for failure
 Wide surface area (50,000 LOC) High MTTR ! All of the bugs we’ve written More opportunities for failure Narrow surface area (< 100 LOC) Low MTTR ! A fraction of the bugs we’ve
 written per release Imagine that we’ll write
  51. 51. Monitoring
  52. 52. Monitoring PHP Warnings Bug Reports and Help Requests
  53. 53. Deploy logs
  54. 54. Post-Mortems
  55. 55. Check in Trigger Feedback Trigger Feedback Approval Approval Smoke Tests User Acceptance! Tests ReleaseDev Team Version Control Build & Unit Tests Automated Acceptance Tests User Acceptance Tests Deploy (Prod) Monitoring and Automated Alerts Continuous Deployment release pipeline Feedback
  56. 56. Dev Pre-Production (“Princess”) Production Check in Trigger Feedback Trigger Feedback Approval ApprovalFeedback Smoke Tests User Acceptance! Tests ReleaseDev Team Version Control Build & Unit Tests Automated Acceptance Tests User Acceptance Tests Deploy (Prod) Monitoring and Automated Alerts Continuous Deployment release pipeline CI
  57. 57. Dev Pre-Production (“Princess”) Production Check in Trigger Feedback Trigger Feedback Approval ApprovalFeedback Smoke Tests User Acceptance! Tests ReleaseDev Team Version Control Build & Unit Tests Automated Acceptance Tests User Acceptance Tests Deploy (Prod) Monitoring and Automated Alerts Continuous Deployment release pipeline CI Approval Approval Feedback Feedback Feedback
  58. 58. Dev Pre-Production (“Princess”) Production Check in Trigger Feedback Trigger Feedback Approval ApprovalFeedback Smoke Tests User Acceptance! Tests ReleaseDev Team Version Control Build & Unit Tests Automated Acceptance Tests User Acceptance Tests Deploy (Prod) Monitoring and Automated Alerts Continuous Deployment release pipeline CI Approval Approval Feedback Feedback Feedback Approval Feedback Approval Feedback
  59. 59. “Allow buttons properly to inherit color from their parent node.”
  60. 60. Five years ago… 2-3 weeks of code changes Release and rollback plans Traffic and infrastructure management (Ops) 6-14 hours
  61. 61. Five years ago… “Deployment Army” Stressful, especially when things go wrong Long days and late nights Scheduled downtime
  62. 62. Why do we do this?
  63. 63. Innovate or die. Resolve scaling hurdles. Mean-time-to-recovery. Healthy and talented engineering team. Stop stressing about releases.
  64. 64. Innovate or die. Resolve scaling hurdles. Mean-time-to-recovery. Healthy and talented engineering team. Stop stressing about releases.
  65. 65. Admin-launch and whitelist Ramp-up public traffic
  66. 66. mainline header_redesign search_filter_custom_orders checkout_blue_button listing_css_refactor
  67. 67. www.etsy.com beta01.etsy.com beta02.etsy.com beta03.etsy.com
  68. 68. www.etsy.com beta01.etsy.com beta02.etsy.com beta03.etsy.com
  69. 69. US","region":"US","detected_currency_code":"USD","detected_language":"en- US","detected_region":"US","accept-languages":"en-US","cdn- provider":"","isMobileDevice":"0","isMobileSupported":"0","isMobileRequestIgnoreCookie":"0" ,"isTabletSupported":"0","isTouch":"0","isEtsyApp":"0","isPreviewRequest":"0","isChromeInst antRequest":"0","isMozPrefetchRequest":"0","listing_ids": [104073511,130604774,159651433,155451607,160523743,124025232,95186610,82967340,114692884,11 4767467,117266897,157579748],"scheduled_modules_content_ids": [10808052776,10256029946],"primary_event":"1",".event_source":"web",".event_logger":"fronte nd","php_ab_test_names":"translation_profiler.profiling;translation_profiler.logging;transl ation_profiler.backend_event_logging;footer_redesign_20131201;international.languages.el;in ternational.languages.ja;international.languages.no;international.languages.pl;internationa l.languages.ro;international.languages.tr;simplified_locale_experience;full_site_ssl;admin_ toolbar;enabled_locale_subdirectories;affiliates.publishing.user_publishers;buyer_invites_r ecipients;home_improvement;home_improvement.new_homepage;authoritative_items;refactored_foo ter;conversations.rejuvination;contextual_homepage_recs.global;css_from_www;shrinkray.css;c srf_nonce_refactor.allow_colon;csrf_nonce_refactor.reverse_order;csrf_nonce_refactor.no_enc Analytics connected to config names
  70. 70. Catapult
  71. 71. Observed impact Time series data for duration of the experiment
  72. 72. Observed impact Time series data for duration of the experiment
  73. 73. Frank Product Manager
  74. 74. “I want to find out whether buyers will favor a single price for the product that includes shipping.” https://www.etsy.com/shop/lucra
  75. 75. Eligibility requirements: - Must be first page of visit - Buyer & seller in same region - etc…
  76. 76. Time: < 8 hours Staff: One ! Design, config flag (disabled), eligibility code in controller, template code, CSS, code review, automated tests, deployed code, config flag enabled.
  77. 77. We do not bundle the item price and shipping cost together today. ! https://www.etsy.com/shop/lucra
  78. 78. Ambitious Product Goal
  79. 79. Ambitious Product Goal Monolithic Building and measuring many things at once.
  80. 80. Ambitious Product Goal Monolithic Building and measuring many things at once. Iterative One thing at a time, our design goal is always in sight.
  81. 81. Time: < 8 hours Staff: One ! Design, config flag (disabled), eligibility code in controller, template code, CSS, code review, automated tests, deployed code, config flag enabled. Deployed Deployed
  82. 82. Is this for me?
  83. 83. http://timothyfitz.com/2009/02/10/continuous-deployment-at-imvu-doing-the-impossible-fifty-times-a-day/ “Maybe this is just viable for a single developer … your site will be down. A lot.”
  84. 84. etsystatus.com
  85. 85. @mikebrittain Very end of 2009 Today DEPLOYMENTSPERDAY APPCODE CONFIGFILES
  86. 86. $1.35 Billion Goods sold in 2013 60+ Million Unique visitors per month ! 175+ Committers, everyone deploys http://www.etsy.com/blog/news/2013/etsy-statistics-december-2012-weather-report/Items by anjaysdesigns, betwixxt, OneStarLeatherGoods, mediumcontrol, TheDesignPallet
  87. 87. Thank you. Mike Brittain Engineering Director, Etsy @mikebrittain mikebrittain.com/talks
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×