Process Improvement Value Stream Service
Delivery Process Program Current state value stream map Committee Eric Switzer Full Full L/T Lead time Projects / Projects / Expedites Expedites P/T Process time H/C Head count EP Business Technical S/R Scrap rate Analysis Analysis Luke Hanson Maya Highsmith Estimates Estimates D Defects and and EP Extra processes requirements requirements M Motion PD Partially done Project Management TS Task switching W Waiting Eric Switzer Business Change Design request Daily Scrum Change Program Management meeting Request QA Control Forms Mx3 build Development Dx3 Develop Technical Speciﬁcation Daily Scrum Deployment web assets meeting QA Meeting Business Systems Jira issues Bugzilla issues Support Release Engineering L/T 8 weeks Web Assets, Deployment P/T 6 weeks Operations ﬂows and Jira issues Document Stan Walters H/C 15 mockups S/R 8% Bob McNulty Wx2 PDx3 Dx2 RT tickets Change TSx5 Dx2 Development Development W Release Control Development Features and ﬁxes Build and deploy Integration testing Build and deploy Set of EP service L/T 10 weeks L/T 1 days L/T 2 day L/T 11 hours Cut project branch branches P/T 6 weeks P/T 2 hours P/T 2 day P/T 8 hours H/C 4 H/C 2 H/C 11 H/C 2 W W D PD Mx7 Mx3 S/R 38% S/R 20% S/R 96% S/R 5% D PDx4 W D L/T 20 minutes D Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez W Deploy to EPx2 P/T 8 minutes Acceptance testing Feature testing Tested Merge project Deploy to Staging Staging testing H/C 2 packages Production branch S/R 0% L/T 5 days L/T 5 weeks L/T 7 days L/T 4 days L/T 10 hours L/T 1 hour Chris Holmes Development P/T 8 hours P/T 3.5 weeks P/T 110 minutes P/T 3.5 days P/T 7 hours P/T 30 minutes Environments H/C 2 H/C 5 H/C 4 H/C 5 H/C 5 H/C 1 Documented S/R 10% S/R 20% S/R 70% S/R 66% S/R 50% S/R 50% test plans Raj Lee Raj Lee Frances Middleton Frances Middleton Allen Cannata Fernando Gomez and cases EP PD Mx3 QA Staging Production Test Requirements Build test plan Build test cases Environments Environment Environment L/T 21 days L/T 8 days L/T 2 weeks P/T 3 days P/T 4 days P/T 8 days H/C 2 H/C 2 H/C 2 S/R 50% S/R 30% S/R 20% Deploy to Performance Luke Hanson Raj Lee Raj Lee Performance testing L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment Process Time 5 weeks 5 weeks 5 hours 3 day 8 hours 6 hours 2 weeks 110 minutes = 59 days 4.5 days 9 hours 3 weeks 5 weeks 2.5 days 2 hours 5 days 2.5 weeks 6.75 days 2.5 days 2 hour Lead Time = 136 days Product Milestone RetailTrader - v133 "Add logic updates " 4 Days Remain 10d total Metrics: 1 week, 1 week historical Value-Creating Time (VCT) 65% +5% Environment: Production Stability Delivery Cycle Time Delivery Support Team Utilization 7 Days 3h, -5h 5 10% Support 85%, -5% -5% Scott, Erica, Reigner,… Provision Lead Time Deployments Work Tasks 2 Hours 15m 8 2 versions 4 environments 4/16 Open -30m +2 environments +1 version 15 Closed Features Provision Scrap Rate Delivery Scrap Rate Build Success 4/5 Lead Time 5% 1/20 +1 20% 2/10 -3 80% -5% 34m avg OK retailtr-v133.war
IntroductionTake a toolchain approach E-commerce
and software-as-a-service business models have matured quickly, but the quality of the web operations that support these businesses has lagged behind. Outages are all too common. High variability and defect rates are bemoaned but have become an accepted reality. Key engineers spend all day (and sometimes all night) mired in deployment issues and bottlenecks. And topping it all off, what tooling that does exist are usually a custom one-offs that are brittle and expensive to maintain. Today’s business of operating software over the Web as a revenue producing service is a ” Today’s business of dramatic departure from the days when software was primarily produced for delivery on operating software over the physical mediums and IT Operations was considered a back-of-the-house support Web as a revenue producing function. Shouldnt we be completely rethinking our tooling and operational capabilities service is a dramatic to match these new innovations? departure from the days when software was primarily In short, we need to get out of Web Operations 1.0 -- mired in legacy tools, outdated produced for delivery on approaches, and low expectations -- and into Web Operations 2.0 where tools and physical mediums...” procedures are built from the ground up for highly efﬁcient, reliable, and agile Industrial Control Automation operations. Runbook Automation Control Eventing, Alarm Mgmt Charting, History, SPC Measurement Instrumentation System + There are multiple factors that go into achieving excellence in Web Operations, but the linchpin that holds it all together is a fully automated provisioning system. In this paper we will be: 1. Deﬁning what we mean by "fully automated provisioning" 2. Explaining why virtualization and cloud computing efforts fail without fully automated provisioning capabilities 3. Proposing a reference open source tool chain for fully automated provisioning 4. Describing a live implementation where a leading online retailer is actively rolling out a fully automated provisioning system using all open source tools Unix Tool Chain 2 Web Ops 2.0: Achieving Fully Automated Provisioning
Design points1. Everything is code2.
Everything is packaged3. Separate code and conﬁguration4. Separate environment-dependent attributes5. Balance distributed vs local orchestration6. Resolve directed vs convergent orchestration7. Integrate application and infrastructure provisioning8. Design for ﬂow not the organization