Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Bringing devopstoanentrenched legacy environmentwithkanban 20160517

203 views

Published on

Innovative Silicon Valley companies like Etsy leverage DevOps and Continuous Delivery practices to achieve new levels of automation and agility, shrinking development lead times and deploying to production many times each day. However, many companies struggle to implement these practices for the legacy systems that run their core business. To make matters worse, the agile community offers relatively little practical guidance for implementing DevOps practices in legacy environments. Fortunately, the Kanban Method provides a practical way to gradually evolve these core systems towards achieving DevOps cost savings and efficiencies—without turning your organization upside down, and even if you don’t have a massive budget.
Through a case study involving a criminal justice system for a US government agency, we will examine how the Kanban method helps us identify and remove the barriers that prevent us from implementing DevOps automation for legacy systems. Just as importantly, Kanban provides the means to measure the efficacy of our efforts, prompting us to course-correct when necessary. Both technology-related and human-related concerns will be addressed. We will review some interesting examples using the Microsoft technology stack, and these lessons can be applied to Java, LAMP, MEAN, or any other set of technologies. The end result is better quality and collaboration and faster delivery of value to our stakeholders. Perhaps it is possible to teach an old dog new tricks, after all.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Bringing devopstoanentrenched legacy environmentwithkanban 20160517

  1. 1. Craeg Strong CTO, Ariel Partners May 17, 2015 San Diego, CA © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394 1 Bringing DevOps to an Entrenched Legacy Environment With Kanban
  2. 2. Software Development since 1988 Large Commercial & Government Projects Agile Coach / (Recent) Kanban Trainer Performance & Scalability Architect Apache Ant Open-Source Contributor New York & Washington DC Area CTO, Ariel Partners AKT, CSM, CSP, CSD, CSPO, PSM, PMI-ACP, PMP, LeSS, SAFe www.arielpartners.com cstrong@arielpartners.com @ckstrong1 Craeg Strong © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  3. 3. Agenda Define: DevOps The Project The Case For DevOps Implementing DevOps For Legacy q Build Automation q Test Automation q Deployment Automation q Configuration As Code Recap and Next Steps © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  4. 4. DEFINE: DEVOPS 4© Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  5. 5. What Is DevOps? 5 DevOps (a clipped compound of "development" and "operations") is a culture, movement or practice that emphasizes the collaboration and communication of both software developers and other information-technology (IT) professionals while automating the process of software delivery and infrastructure changes. DevOps - Wikipedia, the free encyclopedia https://en.wikipedia.org/wiki/DevOps © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  6. 6. DevOps In The Large 6 •Shared Templates & Frameworks •Standards are not Static, Evolve Based on Team Feedback •Multiple Tool chains: Java, .NET, MEAN Tools •Emphasis on Collaboration/Trust •Encourage New Ideas, Limit Downside via Fail-Fast •Blameless Post-Mortems •Servant Leadership •Efficiency vs Thoroughness Trade-off (ETTO Principle) Culture •Cross Training •Emphasis on Continuous Learning •Transition To Management Not Required for Career Advancement •Lightweight, Adaptable •Maximize Throughput, Not Utilization •Tight Feedback Loops •Continuous Service Improvement ProcessOrganization © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  7. 7. Aspects I Will Focus On 7 Build Automation Test Automation Deployment Automation Configuration As Code © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  8. 8. THE PROJECT 8© Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  9. 9. Criminal Justice Program © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  10. 10. System Evolved Over Many Years 10 1990 Pilot DNA Identification Act 1994 1998 Client-Server: VB6 SQL Server 2002 2012 C# VB.Net Multi-Language SOA Re-Architecture VB6 2008 2M Lines of Code © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  11. 11. Change Control Board Meeting Traditional Waterfall Lifecycle M5 M6 M7 M8 M9 M10 M11 M12 Year One M1 M2 M3 M4 Year Two M1 M2 M3 M4 Require- ments Design Code And Unit Test (CUT) System Test GAT UAT PRR PDR CDR TRR TRR OAR © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  12. 12. Federal Contracting Environment BASE YEAR 8/29/2012 – 8/28/2013 2013 2014 20152012 OPTION YEAR 1 8/29/2013 – 8/28/2014 . . . Initial Process Flexibility High Profile Agile Success (Sentinel) Agile Support In Federal Acquisition Regulations High Cost Of Process Change 100% Outsourcing Model Zero Access to Larger Organization © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  13. 13. Challenges Inherent to Legacy Environments 13 q Mission-Critical q Heavily Regulated q Missing or Out of Date q Significant Documentation Requirements q Mostly Manual q System Not Designed For Testability q Resistant to Change q Battle Fatigue q Optimized For Non-Collaboration q May Inhibit Pair Programming q Heavyweight, Prescriptive q Manually-Intensive Reporting q Lengthy Approval Process q Skeptical of Open Source System Criticality Documentation Testing Staff Facility Governance Procurement © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  14. 14. Challenges Implementing Agile Practices In A Legacy Environment 14 q Training, Documentation Assumes New (“Greenfield”) Project q Relatively Little Guidance Regarding Legacy (“Brownfield”) Projects q How Can We Release To Production Every Two Weeks Given A 6-Week Approval Cycle? q How Can We Adopt Two Week Sprints When Regression Testing Consumes The Entire Team For Eight Weeks? q Can’t Fix Everything At Once Agile Training & Literature Agile Practices q Automation Tools Poorly Supported By Older Platforms q Legacy Tends To Be Data-Heavy, Little Tool Support for Data Validation q Business Logic Tends To Be In GUI and Data Layers, Rendering Traditional Unit Testing Useless Agile Toolset © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  15. 15. Why Would Anyone Sign Up For This? 15© Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  16. 16. The Mission 16© Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  17. 17. The Challenge 17© Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  18. 18. The Team 18© Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  19. 19. THE CASE FOR DEVOPS 19© Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  20. 20. Systems Approach To Implementing Kanban (STATIK) 20 1. Understand what makes the service “fit for purpose” 2. Understand sources of dissatisfaction regarding current delivery 3. Analyze sources of and nature of demand 4. Analyze current delivery capability 5. Model the service delivery workflow 6. Identify & define classes of service 7. Design the kanban system 8. Socialize design & negotiate implementation © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  21. 21. Demand Analysis Work Item Type Class Of Service Business User Story Standard Infrastructure User Story Standard ? Tier III / Triage Expedited New Forensic Lab Support Scripts Fixed Date Hot Fix Expedited Escaped Defect Expedited Documentation Deliverable Fixed Date Documentation Rework Expedited Technical Chore Intangible © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  22. 22. Scrum Tool Setup © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  23. 23. Delivery Capability Analysis 0 5 10 15 20 25 Sprint 5: User Story Completion Curve Testers Waiting For Code Limited Time For Testing © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  24. 24. Initial Workflow © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394 Longer Sprint With Bigger Batch-Size Bottleneck Lots of Blocked Issues, Defects, and Rejected Fixes
  25. 25. Sources of Dissatisfaction q Developers Check-in In Code That Breaks the Build q High Concentration of Defects: Fixing One Issue Breaks Two More q Automated Tests Too Difficult To Write q Deployment To Test Environments Lengthy and Error-Prone q Test Set-Up Lengthy and Error-Prone, Tear-Down Usually Skipped q Many False Positive Defects Due To Configuration Issues © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  26. 26. Analyzed Workflow © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  27. 27. BUILD AUTOMATION 27© Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  28. 28. Build Machine Typical Build Automation 28 TFS TeamCity Binary Repository Compile Code Run Tests Package Publish Compile Code Branch B Run Tests Branch B Package Branch B Publish Branch B Compile Code Branch C Run Tests Branch C Package Branch C Publish Branch C © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  29. 29. Build Machine A Build Automation For Legacy System 29 TFS TeamCity Binary Repository Compile Code Run Unit Tests Package Publish Run SQL Tests Drivers, Patches Build Machine B Compile Code Branch B Run Unit Tests Branch B Package Branch B Publish Branch B Run Unit Tests Branch B © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  30. 30. Updated Workflow © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394 “Build Ready” Step Has Been EliminatedBut We Still Have Significant Quality Problems
  31. 31. TEST AUTOMATION 31© Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  32. 32. Typical Test Automation 32 GUI Presentation Logic Server Business Logic DB SQL (CRUD) Test Start Test End üMinimal Setup Required üCode-Only Tests üVery Fast Test Execution üNo Side Effects © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  33. 33. Test Automation For Legacy Systems 33 GUI Presentation-Side Business Logic Server Data Transfer DB SQL Business Logic üSignificant Setup Required üTests Require Database üSlow Test Execution üSignificant Side Effects Test Start Test End © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  34. 34. Build Machine A Incorporating Test Automation Into The DevOps Pipeline 34 TFS TeamCity Binary Repository Compile Code Run Unit Tests Package Publish Run SQL Tests Drivers, Patches Prepare Test DB Run Tests Batch 1 Build Machine B Prepare Test DB Run Tests Batch 2 Build Machine C Prepare Test DB Run Tests Batch 3 Test DB Image © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  35. 35. Updated Workflow © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394 But It Still Takes Too Long Before We Can Test Quality Is Slowly Starting To Improve
  36. 36. DEPLOYMENT AUTOMATION 36© Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  37. 37. Typical Deployment Automation Pipeline 37 TFS TeamCity Version Control Server Build / CI Server Binary Repository Developer Deployment Service Tester © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  38. 38. Deployment Process Deployment Automation For Legacy System 38 Deployment Service Tester TFS Developer PreDeploy.ps1 StopServers UninstallPreviousVersion DeleteOldDatabase Deploy.ps1 RestoreTestDB InstallSoftware © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394 PostDeploy.ps1 AddTestUsers ConfigureSettings CleanupPreviousSession
  39. 39. Updated Workflow © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394 But We Are Still Getting False Positive Defects “Test Setup” Step Has Been Eliminated “Deploy” Step Has Been Streamlined
  40. 40. CONFIGURATION AS CODE 40© Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  41. 41. Deployments Have Two Pieces: Application Code and Operating Platform 41 Operating Platform: Configuration Application: Code Database Matching Engine Application Server Graphical User Interface © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  42. 42. Configuration As Code: Typical Implementation © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394 Web Server Compute Server Database Server Configure TFS Configurations
  43. 43. Configuring The Operating Platform For A Legacy System (1 of 2) No Step Where 1 OS Install 2 Set Hostname & IP Address 3 Set Time zone 4 Install Active Directory Services Production Only 5 Activate Windows 6 Create Messaging & Temp Folders 7 Configure File Share 8 Set Folder Permissions 9 Set Environment Variables 10 Disable Windows Firewall 11 Enable Remote Desktop Dev/Test Only 12 Configure Active Directory Groups © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  44. 44. Configuring The Operating Platform For A Legacy System (2 of 2) No Step Where 13 Install 7-Zip Dev/Test Only 14 Install Octopus Tentacle Dev/Test Only 15 Configure Octopus Tentacle Dev/Test Only 16 Register New Tentacle With Octopus Server Dev/Test Only 17 Install SQL SMO and CLR Packages 18 Install MSMQ 19 Install .NET 3.5 20 Install SQL Server 21 Install IIS 22 Configure FTP/FTPS 23 Configure PKI Certificates 24 Install hMailServer Dev/Test Only, Single Machine © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  45. 45. Evolution Of The Process STEP ONE 100% Manual 300 Page Document STEP THREE The Configurator STEP TWO Partially Automated Individual Scripts © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  46. 46. Enter: The Configurator File Shares Windows SXS Files Third Party Installer Files Test Environment Nodes Targeting File Configurator Script © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394 Configure DevOps Engineer
  47. 47. Updated Workflow © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394 Greatly Reduced Incidence of False Positive Defects Configuring New Environment Reduced Weeks à Hours Validating Existing Environment Reduced Days à Minutes
  48. 48. Key Takeaways 48 1. What DevOps Practices Can Benefit My Project? qBuild Automation qTest Automation qDeployment Automation qConfiguration As Code 2. DevOps for Legacy qChallenging, Yet Transformative 3. Kanban System and STATIK q Lightweight and Effective Guides © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  49. 49. Next Steps 49 qWhere Are The Metrics? qDeepen Kanban Implementation qLeverage Upcoming Punctuation Point qEstablish Real WIP Limits qTimebox à Service Level Agreements qInvestigate Improved Tool Support © Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394
  50. 50. 50© Copyright Ariel Partners 2016 *sales@arielpartners.com ((646) 467-7394

×