Lean & Agile Methods & Frameworks: Perspectives on Kanban for IT
Perspectives on Kanban for ITDr. David F. Rico, PMP, ACP, CSMTwitter: @dr_david_f_ricoWebsite: http://www.davidfrico.comLinkedIn: http://www.linkedin.com/in/davidfricoFacebook: http://www.facebook.com/profile.php?id=1540017424Dave’s Agile Articles: http://davidfrico.com/agile-message.docLean & AgileMethods & Frameworks
Author Background DoD contractor with 28+ years of IT experience B.S. Comp. Sci., M.S. Soft. Eng., & D.M. Info. Sys. Large gov’t projects in U.S., Far/Mid-East, & Europe2 Published six books & numerous journal articles Adjunct at George Wash, UMBC, UMUC, Argosy Agile Program Management & Lean Development Specializes in metrics, models, & cost engineering Six Sigma, CMMI, ISO 9001, DoDAF, & DoD 5000 Cloud Computing, SOA, Web Services, FOSS, etc.
Today’s Whirlwind Environment3OverrunsAttritionEscalationRunawaysCancellationGlobalCompetitionDemandingCustomersOrganizationDownsizingSystemComplexityTechnologyChangeVagueRequirementsWork LifeImbalanceInefficiencyHigh O&MLower DoQVulnerableN-M BreachReducedIT Budgets81 MonthCycle TimesRedundantData CentersLack ofInteroperabilityPoorIT SecurityOverburdeningLegacy SystemsObsoleteTechnology & SkillsPine, B. J. (1993). Mass customization: The new frontier in business competition. Boston, MA: Harvard Business School Press.Pontius, R. W. (2012). Acquisition of IT: Improving efficiency and effectiveness in IT acquisition in the DoD. Second AnnualAFEI/NDIA Conference on Agile in DoD, Springfield, VA, USA.
Traditional Projects4 Big projects result in poor quality and scope changes Productivity declines with long queues/wait times Large projects are unsuccessful or canceledJones, C. (1991). Applied software measurement: Assuring productivity and quality. New York, NY: McGraw-Hill.Size vs. QualityDefectDensity0.003.206.409.6012.8016.000 2 6 25 100 400Lines of Code (Thousands)Size vs. ProductivityCodeProductionRate0.001.002.003.004.005.000 2 6 25 100 400Lines of Code (Thousands)Size vs. Requirements GrowthPercentage0%8%16%24%32%40%0 2 6 25 100 400Lines of Code (Thousands)Size vs. SuccessPercentage0%12%24%36%48%60%0 2 6 25 100 400Lines of Code (Thousands)
Global Project Failures5Standish Group. (2010). Chaos summary 2010. Boston, MA: Author.Sessions, R. (2009). The IT complexity crisis: Danger and opportunity. Houston, TX: Object Watch. Challenged and failed projects hover at 67% Big projects fail more often, which is 5% to 10% Of $1.7T spent on IT projects, over $858B were lost16% 53% 31%27% 33% 40%26% 46% 28%28% 49% 23%34% 51% 15%29% 53% 18%35% 46% 19%32% 44% 24%33% 41% 26%0% 20% 40% 60% 80% 100%199419961998200020022004200620082010YearSuccessful Challenged Failed$0.0$0.4$0.7$1.1$1.4$1.82002 2003 2004 2005 2006 2007 2008 2009 2010Trillions(USDollars)Expenditures Failed Investments
Requirements Defects & Waste6Sheldon, F. T. et al. (1992). Reliability measurement: From theory to practice. IEEE Software, 9(4), 13-20Johnson, J. (2002). ROI: Its your job. Extreme Programming 2002 Conference, Alghero, Sardinia, Italy. Requirements defects are #1 reason projects fail Traditional projects specify too many requirements More than 65% of requirements are never used at allOther 7%Requirements47%Design28%Implementation18%DefectsAlways 7%Often 13%Sometimes16%Rarely19%Never45%Waste
What is Agility? A-gil-i-ty (ə-ji-lə-tē) Property consisting of quickness,lightness, and ease of movement; To be very nimble The ability to create and respond to change in order toprofit in a turbulent global business environment The ability to quickly reprioritize use of resources whenrequirements, technology, and knowledge shift A very fast response to sudden market changes andemerging threats by intensive customer interaction Use of evolutionary, incremental, and iterative deliveryto converge on an optimal customer solution Maximizing BUSINESS VALUE with right sized, just-enough, and just-in-time processes and documentationHighsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.7
What are Agile Methods?8 People-centric way to create innovative solutions Product-centric alternative to documents/process Market-centric model to maximize business valueAgile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.orgRico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.Rico, D. F. (2012). Agile conceptual model. Retrieved February 6, 2012, from http://davidfrico.com/agile-concept-model-1.pdfCustomer CollaborationWorking SoftwareIndividuals & InteractionsResponding to Changevaluedmore thanvaluedmore thanvaluedmore thanvaluedmore thanContractsDocumentationProcessesProject Plans Frequent comm. Close proximity Regular meetings Multiple comm. channels Frequent feedback Relationship strength Leadership Boundaries Empowerment Competence Structure Manageability/Motivation Clear objectives Small/feasible scope Acceptance criteria Timeboxed iterations Valid operational results Regular cadence/intervals Org. flexibility Mgt. flexibility Process flexibility System flexibility Technology flexibility Infrastructure flexibility Contract compliance Contract deliverables Contract change orders Lifecycle compliance Process Maturity Level Regulatory compliance Document deliveries Document comments Document compliance Cost Compliance Scope Compliance Schedule Compliance
NetworkComputerOperating SystemMiddlewareApplicationsAPIsGUIHow Agile Works Agile requirements implemented in slices vs. layers User needs with higher business value are done first Reduces cost & risk while increasing business success9Shore, J. (2011). Evolutionary design illustrated. Norwegian Developers Conference, Oslo, Norway.Agile Traditional1 2 3 Faster Early ROI Lower Costs Fewer Defects Manageable Risk Better Performance Smaller Attack SurfaceLate No Value Cost Overruns Very Poor Quality Uncontrollable Risk Slowest Performance More Security Incidents Seven Wastes1.Rework2.Motion3.Waiting4.Inventory5.Transportation6.Overprocessing7.OverproductionMINIMIZES MAXIMIZES JIT, Just-enough architecture Early, in-process system V&V Fast continuous improvement Scalable to systems of systems Maximizes successful outcomes Myth of perfect architecture Late big-bang integration tests Year long improvement cycles Breaks down on large projects Undermines business success
Thousands of TestsContinuously ExecutedNo More Late BigBang IntegrationAgile Development Model User needs designed & developed one-at-a-time Changes automatically detected, built, and tested System fully tested and deployed as changes occur10Humble, J., & Farley, D. (2011). Continuous delivery. Boston, MA: Pearson Education.Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration. Boston, MA: Addison-Wesley.BuildIntegrationServerVersionControlServerBuildScriptsUsesWatchesBuildStatusProvidesDeveloper ADeveloper BDeveloper CCommitsChangesCommitsChangesCommitsChangesBuildsDatabaseAnalysisTestingReportingDocumentationDeploymentEarly, Automated, Fast,Efficient, & RepeatableConstant ReadinessState & CM ControlLean, Waste Free, Low WIP,No Deadlocked Test QueuesRapidly & SuccessfullyDev. Complex Systems
How Do Lean & Agile Intersect?11 Agile is naturally lean and based on small batches Agile directly supports six principles of lean thinking Agile may be converted to a continuous flow systemWomack, J. P., & Jones, D. T. (1996). Lean thinking: Banish waste and create wealth in your corporation. New York, NY: Free Press.Reinertsen, D. G. (2009). The principles of product development flow: Second generation lean product development. New York, NY: Celeritas.Reagan, R. B., & Rico, D. F. (2010). Lean and agile acquisition and systems engineering: A paradigm whose time has come. DoD AT&L Magazine, 39(6). Economic ViewDecentralizationFast FeedbackControl Cadence& Small BatchesManage Queues/Exploit VariabilityWIP Constraints& KanbanFlow PrinciplesAgile ValuesCustomerCollaborationEmpoweredTeamsIterativeDeliveryRespondingto ChangeLean PillarsRespectfor PeopleContinuousImprovementCustomer ValueRelationshipsCustomer PullContinuous FlowPerfectionValue StreamLean Principles Customer relationships, satisfaction, trust, and loyalty Team authority, empowerment, and resources Team identification, cohesion, and communicationLean & Agile Practices Product vision, mission, needs, and capabilities Product scope, constraints, and business value Product objectives, specifications, and performance As is policies, processes, procedures, and instructions To be business processes, flowcharts, and swim lanes Initial workflow analysis, metrication, and optimization Batch size, work in process, and artifact size constraints Cadence, queue size, buffers, slack, and bottlenecks Workflow, test, integration, and deployment automation Roadmaps, releases, iterations, and product priorities Epics, themes, feature sets, features, and user stories Product demonstrations, feedback, and new backlogs Refactor, test driven design, and continuous integration Standups, retrospectives, and process improvements Organization, project, and process adaptability/flexibility
What is Lean Development? Lean (lēn): Thin, slim, slender, narrow, adequate,or just-enough; Without waste A customer-driven product development process thatdelivers the maximum amount of business value An economical way of planning and managing thedevelopment of complex new products and services A product development process that is free of excesswaste, capacity, and non-value adding activities Just-enough, just-in-time, and right-sized productdevelopment processes, documentation, and tools A product development approach that is adaptable tochange in customer needs and market conditions12Womack, J. P., & Jones, D. T. (1996). Lean thinking: Banish waste and create wealth in your corporation. New York, NY: Free Press.Liker, J. K. (2004). The toyota way: 14 management principles from the world’s greatest manufacturer. New York, NY: McGraw Hill.Larman, C., & Vodde, B. (2008). Scaling lean and agile development: Thinking and organizational tools for large-scale scrum. Boston, MA: Addison-Wesley.
Lean Thinking13 Term coined by John Krafcik of MIT in 1988 Taiichi Ohno of Toyota is credited with its ideas Toyota Production System was adapted from FordWomack, J. P., & Jones, D. T. (1996). Lean thinking: Banish waste and create wealth in your corporation. New York, NY: Free Press.Liker, J. K. (2004). The toyota way: 14 management principles from the world’s greatest manufacturer. New York, NY: McGraw Hill.Larman, C., & Vodde, B. (2008). Scaling lean and agile development: Thinking and organizational tools for large-scale scrum. Boston, MA: Addison-Wesley.
Lean Six Sigma14 Created in late 1990s by Allied Signal and Maytag Combination of Six Sigma and Lean Thinking Focuses on eliminating waste vs. variationGeorge, M. L. (2002). Lean six sigma: Combining six sigma quality with lean speed. New York, NY: McGraw-Hill.
Lean Product Development15 Lean product development emerged in the 1980s Adaptation of Toyota Production System (TPS) “Toyota [New] Product Development System”Clark, K. B., & Fujimoto, T. (1991). Product development performance: Strategy, organization, and management in the world auto industry. Boston, MA: Harvard Business School Press.Lean DevelopmentFrequent set-up changesLean ManufacturingShort manufacturing throughput timeReduced work-in-process inventory betweenmanufacturing stepsFrequent transfer of small batches of parts betweenmanufacturing stepsReduced inventory requires slack resources andmore information flow between stepsAdaptability to changes in volume, product mix, andproduct designBroad task assignments for production workersgives higher productivityFocus on quick problem solving and continuousprocess improvementSimultaneous improvement in quality, delivery time,and manufacturing productivityFrequent product changesShort development timeReduced information inventory betweendevelopment stepsFrequent transfer of preliminary informationbetween development stepsReduced development time requires slackresources and information flow between stagesAdaptability to changes in product design,schedule, and cost targetsBroad task assignments for engineers(developers) gives higher productivityFocus on frequent incremental innovation andcontinuous product and process improvementSimultaneous improvement in quality,development time, and development productivity
Lean Systems Engineering16 Origin in MIT Lean Aerospace Initiative in 1992 Lean Systems Engineering WG formed in 2006 Lean Enablers for Systems Engineering in 2009INCOSE. (2009). Lean enablers for systems engineering. Retrieved October 20, 2009, from http://www.incose.org/practice/techactivities/wg/leansewg.ValueMap theValueStreamFlowPullPerfectionRespect forPeopleCustomer involvementStreamline developmentDeconflict suppliersEstablish metricsCommunicate effectivelyAchieve smooth flowEnsure program visibilityUse leadPull tasks as-neededAppoint a chiefEliminate wastePerform continuous improvementCreate a learning environmentTreat people as valuable assetsRequirements engineeringBusiness value analysisProgram planningMap value streamsFrontload programProgram monitoringClarify requirementsFrontload architectureCoordinate activitiesTailor programContinuous improvementStrive for excellencePerform lessons learnedDevelop communication planHuman resourcesRespect all peopleStrive for technical excellence
What is Kanban? Kan-ban (kæn-bæn): Signboard, billboard, signalcards; Lean, just-in-time system of production A lean and just-in-time manufacturing process forregulating the flow of production based on demand A pull-system philosophy of customized production vs.a push system of mass-market manufacturing A set of principles for creating a lean, efficient, andwaste-free product flow by limiting work-in-process Use of simple organizational policy changes resultingin order-of-magnitude performance improvements Framework for optimizing workflow that maximizesefficiency, product quality, and customer satisfaction17Kniberg, H., & Skarin, M. (2010). Kanban and scrum: Making the most of both. Toronto, ON: C4 Media, Inc.Ladas, C. (2008). Scrumban: Essays on kanban systems for lean software development. Seattle, WA: Modus Cooperandi.Anderson, D. J. (2010). Kanban: Successful evolutionary change for your technology business. Sequim, WA: Blue Hole Press.
Kanban for IT Projects18 Adapted to IT by Dave Anderson in 2006 Activities, buffers, queues, WIP limits, tasks, etc. Lean, JIT pull/demand system leading to high qualityAnderson, D. J. (2010). Kanban: Successful evolutionary change for your technology business. Sequim, WA: Blue Hole Press.
Kanban Goals19 Kanban initially seeks to change as little as possible Change without resistance is the first Kanban goal Focus on improving quality, lead time and moraleGoal 2Goal 3Goal 4Goal 5Goal 6Goal 7Goal 8Deliver high product quality (to build stakeholder trust)Reduce long lead times (and stabilize them)Achieve sustainable pace (work-life balance)Provide process slack (for process improvement)Simplify workload prioritization (of customer needs)Provide transparency (into design and operations)Strive for process maturity (to improve performance)Goal 1 Optimize existing processes (rather than change them)Anderson, D. J. (2010). Kanban: Successful evolutionary change for your technology business. Sequim, WA: Blue Hole Press.
Kanban Recipe for Success20 Based on principles for product development flow Uses operations and mathematical queue theory Pragmatic operating principles for developmentFocus on Quality Reduce WIP Deliver Often Balance Demand Prioritize Attack Variability Walkthroughs Inspections Technical reviews Peer reviews Pair programming Test driven design Continuous integration Design patterns Refactoring Design simplicity Usability engineering Formal methods Process flowcharts Workflow analysis Kanban boards Limit work tasks Limit queues Limit buffers Limit backlogs Simple prioritization Adequate resources Process automation Policy statements Simplify process Short releases Short increments Short iterations Small releases Frequent releases Small batch sizes Customer collaboration Developer collaboration Ample communication Frequent builds Deploy often Automatic updates Regulate inputs Identify bottlenecks Create slack Limit work-in-process Create pull system Focus on precision Focus on quality Take pride in work Improve morale Learn new skills Obtain training Continuously improve Prioritize inputs Business focus- Business value focus Influence prioritization Stabilize process Build stakeholder trust Perform risk analysis Analyze demand Evaluate size Evaluate complexity Market forecasting Technology analysis Work item size Work item type mix Service class mix Irregular flow Rework Ambiguous reqmnts. Expedited requests Environment avail. Market fluctuations Coordination Technological change Skill/experience mixAnderson, D. J. (2010). Kanban: Successful evolutionary change for your technology business. Sequim, WA: Blue Hole Press.
Value Stream Mapping21 Start by flow-charting the as-is product workflow Add buffers and queues one feels are necessary Add WIP limits to buffers, queues, and activityAnderson, D. J. (2010). Kanban: Successful evolutionary change for your technology business. Sequim, WA: Blue Hole Press.
22Traditional vs. Agile Cumulative FlowWork(Story,Point,Task)orEffort(Week,Day,Hour)Time Unit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.)Work(Story,Point,Task)orEffort(Week,Day,Hour)Time Unit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.)Traditional Cumulative Flow Lean & Agile Cumulative FlowAnderson, D. J. (2004). Agile management for software engineering. Upper Saddle River, NJ: Pearson Education.Anderson, D. J. (2010). Kanban: Successful evolutionary change for your technology business. Sequim, WA: Blue Hole Press. High work-in-process leads to longest lead times Low work-in-process greatly reduces lead times Results in better customer trust and satisfactionWork-in-Process
Scaled Lean & Agile FrameworkBeck, K., & Fowler, M. (2001). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.Highsmith, J. A. (2010). Agile project management: Creating innovative products. Boston, MA: Pearson Education.Larman, C., & Vodde, B. (2010). Practices for scaling lean and agile development. Boston, MA: Addison-Wesley.Leffingwell, D. (2011). Agile software requirements: Lean requirements practices for teams, programs, and the enterprise. Boston, MA: Pearson Education. Begins with a high-level product vision/architecture Continues with needs development/release planning Includes agile delivery teams to realize business value23
Lean & Agile Hybrid Methods24Shore, J., & Warden, S. (2008). The art of agile development. Sabastopol, CA: OReilly Media.Cockburn, A. (2007). Agile software development: The cooperative game. Boston, MA: Addison-Wesley.Shalloway, A., Beaver, G., & Trott, J. R. (2010). Lean-agile software development. Boston, MA: Addison-Wesley. Scrum is a basic agile project management model Scrum does not specify other critical functions Scrum hybrids include necessary practicesEnterprisePortfolio analysisEarned value mgt.Enterprise arch.GovernanceScrumSprint PlanningDaily ScrumSprint ReviewRetrospectiveXPRelease planningPair programmingContinuous integ.Open workspacesOpen SourceEnvironmentsComponentsFrameworksOperating systemsOtherUser exp.Security eng.Reliability eng.Safety analysis
Lean & Agile Recap Agile methods DON’T mean deliver it now & fix it later Lightweight, yet disciplined approach to development Reduced cost, risk, & waste while improving quality25Rico, D. F. (2012). What’s really happening in agile methods: Its principles revisited? Retrieved June 6, 2012, from http://davidfrico.com/agile-principles.pdfRico, D. F. (2012). The promises and pitfalls of agile methods. Retrieved February 6, 2013 from, http://davidfrico.com/agile-pros-cons.pdfRico, D. F. (2012). How do lean & agile intersect? Retrieved February 6, 2013, from http://davidfrico.com/agile-concept-model-3.pdfWhat How ResultFlexibility Use lightweight, yet disciplined processes and artifacts Low work-in-processCustomer Involve customers early and often throughout development Early feedbackPrioritize Identify highest-priority, value-adding business needs Focus resourcesDescope Descope complex programs by an order of magnitude Simplify problemDecompose Divide the remaining scope into smaller batches Manageable piecesIterate Implement pieces one at a time over long periods of time Diffuse riskLeanness Architect and design the system one iteration at a time JIT waste-free designSwarm Implement each component in small cross-functional teams Knowledge transferCollaborate Use frequent informal communications as often as possible Efficient data transferTest Early Incrementally test each component as it is developed Early verificationTest Often Perform system-level regression testing every few minutes Early validationAdapt Frequently identify optimal process and product solutions Improve performance
Conclusion26 Agility is the evolution of management thought Confluence of traditional and non-traditional ideas Improve performance by over an order of magnitude“The world of traditional methods belongs to yesterday”“Don’t waste your time using traditional methods on 21st century projects”Agile methods are …Systems development approachesNew product development approachesExpertly designed to be fast and efficientIntentionally lean and free of waste (muda)Systematic highly-disciplined approachesCapable of producing high quality systemsRight-sized, just-enough, and just-in-time tools Scalable to large, complex mission-critical systems Designed to maximize business value for customersWysocki, R.F. (2010). Adaptive project framework: Managing complexity in the face of uncertainty. Boston, MA: Pearson Education.
Books on ROI of SW Methods Guides to software methods for business leaders Communicates business value of software methods Rosetta stones to unlocking ROI of software methods http://davidfrico.com/agile-book.htm (Description) http://davidfrico.com/roi-book.htm (Description)27