Recommended
Recommended
More Related Content
Similar to Getting People Outside with Continuous Delivery
Similar to Getting People Outside with Continuous Delivery (20)
Recently uploaded
Recently uploaded (20)
Getting People Outside with Continuous Delivery
- 1. GETTING PEOPLE OUTSIDE WITH CONTINUOUS DELIVERY Todd Wilson Director, Platform Engineering twilson@rei.com | @t1wilson
- 2. Agenda • About REI • Our Digital Moments • Continuous Delivery Journey • Organizational Shifts • Results • Lessons Learned
- 3. About REI • Seattle-based outdoor gear retailer founded in 1938 • Largest co-op in nation with 6M+ active members • 12000+ employees with 400 in technology • $2.5B in sales in 2016, +5.5% • $194M returned in dividends and CC rebates • $9.3M contributed to non-profits and National Parks Foundation
- 5. A life OUTDOORS IS A life WELL LIVED
- 6. Gear
- 8. • 6 million people opted outside, up from 1.4 in 2015 • 700+ non-profit, community and corporate partners including Subaru, Google and National Park Service • 3.3 billion media impressions
- 11. More than 1,000 events designed for women in 2017 Women-focused storytelling $1M commitment to nonprofits creating opportunity for women outdoors High-performance gear and apparel for women
- 13. ~3 Years REI.com releases every 6 weeks every 2 weeks every 1 week daily continuous deployment 2 Years Today
- 14. Build Pipeline
- 15. Proprietary testing tool to open source Testing SHARED OWNERSHIP OF REGRESSION SUITE REGRESSION FROM 96 HOURS TO 20 MIN
- 16. From release branches to releasing from master
- 17. Feature toggles
- 18. CONFIDENTIAL | FOR INTERNAL USE ONLY 18 Release veto Anyone could stop a release Then just committers
- 19. Monitoring
- 21. Communication Team collaboration, alerts, notifications, bots
- 22. CONFIDENTIAL | FOR INTERNAL USE ONLY 22 Change management Code deployments are decoupled from feature releases Code deployments are treated as standard changes. Pre-approved and change records auto inserted. Major feature releases go through CAB for visibility.
- 24. Results
- 25. Reduced time-to-market New Adventures website delivered in 6 months
- 26. Replaced Outlet with REI Garage in 9 months
- 30. Lessons Learned • Leverage Value Stream Mapping to identify bottlenecks • Collect pipeline metrics for continuous improvement • Educate your technology and business partners • Iterate and celebrate each success
Editor's Notes
- At the time we had an offshore team from one of our partners who was responsible for maintaining our regression test suite. They used a commercial tool which was completely different than any of the tools and languages our teams were using. The tests were brittle and unreliable. Decided to migrate the test suite to Selenium since it was Java based but still relied on that same team. We found that all we did was move bad tests from one tool to another. We then decided we needed to bring the tests back in house. So we began investing in the test engineering role on each of the teams and assigned ownership to rewrite the tests in their areas.
- As part of the change we moved away from teams developing on branches to trunk based development. This certainly got easier with our migration to git but it was also made easier by our existing system for managing runtime configurations.
- We had a system in place but we have enhanced it significantly since this capability has become such a critical component of our CD practice. We would love to open source some day if we can find the time
- Another concept we introduced was a release veto. Each time we made a new leap whether it be to weekly, daily or continuous releases there has been a level of anxiety and fear which was natural. In order to help with the transition we introduced the notion of a release vote, which is really a veto. Everyone who was interested could subscribe to the notification that a realease was ready to be staged. This notification had a link that would allow someone to veto or cancel the release within a defined window of time. Over time we have transitioned to where it’s primarily just the engineers with commits in the build who can release.
- As with all successful CD implementations monitoring is a critical component. We invested in deploying several new capabilities for monitoring production. We invested in commercial tools like New Relic and deployed open source tools like Logstash, StatsD and Graphite. The tools gave us a level of visibility we hadn’t had before.
- New communication tools have dramatically increased collaboration amongst the organization as well as reduced the amount of email through consolidation of
- Early on invested in a dedicated team to build these tools and capabilities. The team is a multiplier that has a dramatic affect on the organization.
- CD has been awesome for business agility but we’ve also seen an increase in stability and quality. You can see the number of production incidents have decreased but we’ve also seen a reduction in downtime for major incidents. I haven’t’ checked out downtime but we were reaching 99.99% for the year. With the confidence in our deployments we partnered with the business to eliminate our holiday code freeze. We maintained a modified calendar and carefully scrutinized the type of changes to deploy but ultimately this gives our business the ability to react and optimized during our highest volume season. This is pretty unheard of for a retailer. As I move into our steps to get to CD it’s important to note that you likely won’t hear anything new. All the techniques we’ve used are well documented and known but I want to demonstrate that it is possible in a traditional environment. For us the journey started back around 2010. We had adopted Scurm and got pretty good at it over the course of a year. But it became clear that we weren’t maximizing ability to develop as we were still deploying every few weeks or even a month. It was at that point another manager and I decided to find a better way. It just so happened my colleague was in dev and I had recently moved from dev to manage a team in infrastructure.