Your SlideShare is downloading. ×
Managing Experimentation in a Continuously Deployed Environment
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Managing Experimentation in a Continuously Deployed Environment

355
views

Published on

Video and slides synchronized, mp3 and slide download available at http://bit.ly/12Xi2lC. …

Video and slides synchronized, mp3 and slide download available at http://bit.ly/12Xi2lC.

Wil Stuckey explains how Etsy manages to deploy nearly ~10,000 changes in one year, and how they run A/B experiments in the midst of continual code change.Filmed at qconnewyork.com.

Wil Stuckey is Software Engineer at Etsy.

Published in: Technology, Education

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
355
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Wil Stuckey @quiiver
  • 2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /etsy-deploy
  • 3. Presented at QCon New York www.qconnewyork.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
  • 4. 25 Million Members
  • 5. 25 Million Members 900,000 Shops `
  • 6. 25 Million Members 900,000 Shops 1.4B Page Views / Month
  • 7. Managing Experimentation
  • 8. Managing Experimentation
  • 9. Managing Launches
  • 10. 1. Experiments 2. Rampups 3. Communications
  • 11. Life Before Continuous Deployments
  • 12. Initial Idea & Planning
  • 13. Geocities-ize
  • 14. Initial Idea & Planning Design & Development
  • 15. Initial Idea & Planning Deploy & Launch! Design & Development
  • 16. Pain?
  • 17. Testing?
  • 18. Communication?
  • 19. How can we do better?
  • 20. Living in a Continuous Deployment World
  • 21. Total Deploys: 4,040 Total Commits: 45,447 Total Unique Authors: 220 2013 ... So far
  • 22. Total Deploys: 4,040 Total Commits: 45,447 Total Unique Authors: 220 Average Deploys per day: 35.75 Average Authors per deploy: 7.01 Average Commits per deploy: 11.25 2013 ... So far
  • 23. That’s a lot of people
  • 24. We are only halfway through the year!
  • 25. CHAOS
  • 26. Config System $server_config['my_awesome_feature'] = [ 'enabled' => 10, 'admin' => 'on', 'groups' => [1234], 'users' => ['wstuckey', 'dmckinley'], ];
  • 27. Branching in Code if (Feature::isEnabled('my_awesome_feature')) { // display my_awesome_feature... } else { // continue on }
  • 28. Typical Launch Cycle
  • 29. Initial Idea & Planning
  • 30. Initial Idea & Planning Design & Development Launch!
  • 31. Initial Idea & Planning Design & Development Launch! Internal Admin Launch
  • 32. Initial Idea & Planning Design & Development Launch! Internal Admin Launch Public Prototype
  • 33. Initial Idea & Planning Design & Development Launch! Internal Admin Launch Public Prototype 50% Experiment
  • 34. • Small iterative changes
  • 35. • Small iterative changes • Product exists in prod as soon as possible
  • 36. • Small iterative changes • Product exists in prod as soon as possible • Can be available for small groups for testing and feedback.
  • 37. • Small iterative changes • Product exists in prod as soon as possible • Can be available for small groups for testing and feedback. • Easy to communicate milestones because they happen as part of the development process.
  • 38. Because of the constant change you have to constantly communicate.
  • 39. Start Simple
  • 40. Wiki + Email
  • 41. • Works great in the beginning Wiki
  • 42. • Works great in the beginning • Quickly becomes overwhelming as you increase your concurrent launches. Wiki
  • 43. • Works great in the beginning • Quickly becomes overwhelming as you increase your concurrent launches. • Requires manual labor to keep stakeholders informed Wiki
  • 44. FRUSTRATION
  • 45. We can do better!
  • 46. Frank Harris @hirefrank
  • 47. Launch Calendar
  • 48. Launch Calendar • Simple web app • Collect structured meta-data about the launch • Central location for upcoming, current and past launch information
  • 49. Launch Calendar Collect launch specific meta-data
  • 50. Launch Calendar • Owner • Launch Date • Internal Launch Date • Team • Expected Reaction Collect launch specific meta-data
  • 51. Launch Calendar ✉ Make your email work for you
  • 52. ✉ Launch Calendar • Automated emails to product owners around deadlines • Daily emails to product groups containing upcoming and recently launched launches Make your email work for you
  • 53. Launch Calendar Make your email work for you ✉
  • 54. Launch Calendar • Highly utilized across the organization • During the year+ that it was running we recorded and tracked over 275 launches How did it go?
  • 55. “This beats the crap out of the wiki page in every conceivable way.” - real feedback
  • 56. What could we do better?
  • 57. What would a more integrated product look like?
  • 58. Can we automate more?
  • 59. We can do better!
  • 60. Catapult A unified launch management tool.
  • 61. “I believe that by changing the ‘add to cart’ button to blue we will increase conversion rate.”
  • 62. Catapult
  • 63. $server_config['my_awesome_button'] = [ 'enabled' => 50, 'admin' => 'on', ]; Remember this?
  • 64. Hack, Deploy, Hack, Hack Deploy!
  • 65. Integration is awesome.
  • 66. Communication is awesome.
  • 67. Start Simple.
  • 68. Build process that enable you to Ship.
  • 69. Never stop deploying!
  • 70. Thank You!
  • 71. Wil Stuckey @quiiver