Managing Technical Debt
•
1 like•349 views
This talk was presented at AgileDC2018 Abstract: Is your team constantly missing delivery dates? Is the velocity decreasing from sprint to sprint while the development costs are rising? Are customers complaining about the increasing number of bugs and the long time it takes to add new features? These are all signs that you are mired in technical debt and probably on your way to bankruptcy or a complete system rewrite. Technical debt is inevitable, whether intentional or unintentional. However, not managing technical debt can paralyze your organization. Fadi Stephan expands on the technical debt metaphor and introduces a technical debt management plan that enables executives and teams to make prudent decisions on code quality and technical debt. Come learn how to measure the quality of your code base and determine the amount of your debt.
Recommended
Recommended
More Related Content
Similar to Managing Technical Debt
Similar to Managing Technical Debt (20)
More from Fadi Stephan
More from Fadi Stephan (20)
Recently uploaded
Recently uploaded (20)
Managing Technical Debt
- 1. Managing Technical Debt Managing Technical Debt Fadi Stephan @FadiStephan
- 3. While we wait for the session to start, meet your neighbors and discuss … • Do you think you are currently suffering from technical debt? • How do you know? What are the indications that you are or are not?
- 4. @FadiStephan Fadi Stephan 20+ years of experience in software development Technology Consultant, Agile Coach and Trainer Co-organizer of the DC Scrum User Group (DCSUG) www.scrumalliance.org/community/profile/fstephan @FadiStephan
- 5. @FadiStephan
- 6. @FadiStephan
- 7. @FadiStephan
- 8. @FadiStephan
- 11. Rigidity
- 14. Immobility
- 15. Viscosity
- 17. Deadline
- 18. Broken Window
- 22. Bad Design
- 23. Poor Skills
- 25. Technical Debt “Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite... The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt.” - Ward Cunningham
- 26. Technical Debt Metaphor “Neglecting the design is like borrowing money” “Refactoring, it's like paying off the principal debt” “Developing slower because of this debt is like paying interest on the loan” “Every minute spent on not-quite-right code counts as interest on that debt”
- 27. Principal Interest Technical Debt Quick and dirty design results in
- 32. Which one will you choose? 1. Quick and Dirty 2. Clean
- 33. Home or car loan
- 35. Types of Debt • Unintentional • Intentional – Short term & focused – Short term & unfocused – Long term • Only short term focused debt & long term debt are “good” debt forums.construx.com/blogs/stevemcc/archive/2007/11/01/technical-debt-2.aspx
- 41. Technical Debt Backlog Story Dirty Estimate Clean Estimate On Going Impact Date … 3 8 H 2/5/2018 … 1 5 M 2/10/2018 … 3 13 L 2/11/2018
- 42. Debt Penalty Story Estimate Debt Area With Interest Penalty … 3 H 8 … 8 M 13 … 13 L 13
- 45. Technical Debt = #items * #hours/item * $/hr Monetize the Debt
- 46. Technical Debt Plugin Debt(in man days) = cost_to_fix_duplications + cost_to_fix_violations + cost_to_comment_public_API + cost_to_fix_uncovered_complexity + cost_to_bring_complexity_below_threshold + cost_to_cut_cycles_at_package_level
- 50. How much debt is too much debt?
- 51. Paying Down The Debt
- 53. Pay debt with high interest rate 1st
- 55. Approach • Have a technical debt reduction sprint immediately after a release (better than nothing) • Have a technical debt reduction sprint once we reach a certain limit (better than nothing) • Rotate dedicated members to work on reducing technical debt (good) • Dedicate 10% of each sprint to reducing technical debt (better) • Reduce technical debt by story (best)
- 56. Summary Managing technical debt requires that we make prudent and deliberate decision on design & quality
- 57. Summary Provide transparency by 1. Registering any new debt 2. Assessing existing debt
- 58. Summary Inspect by 1. Monetizing the debt 2. Establishing a debt limit 3. Monitor trends
- 59. Summary Adapt by 1. Paying down the debt focusing on high interest rate 1st. 2. Starting with what you know. Train for the rest 3. Continuously monitor the debt
- 61. Acknowledgement Robert Martin Martin FowlerIsrael GatSteve McConnell
- 62. References • Design Principles and Design Patterns - Robert Martin • Design Stamina Hypothesis - martinfowler.com/bliki/DesignStaminaHypothesis.html • Technical Debt Quadrant - martinfowler.com/bliki/TechnicalDebtQuadrant.html • The Agile Triangle – theagileexecutive.com/tag/the-agile-triangle/ • Technical Debt Assessment and Reduction – theagileexecutive.com/category/technical-debt/ • Technical Debt, Cutter IT Journal October 2010 - www.cutter.com
- 63. References • Technical Debt A Perspective for Manager – www.infoq.com/articles/technical-debt-levison • Managing Technical Debt - blogs.versionone.com/agile_management/2011/07/11/managing-technical-debt/ • What Testers Can Do About Technical Debt - www.stickyminds.com/sitewide.asp?ObjectId=3629 • Climb Out of Technical Debt – www.ayeconference.com/climboutoftechnicaldebt/ • Don't Live with Broken Windows– www.artima.com/intv/fixit.html
- 64. References • Technical Debt - forums.construx.com/blogs/stevemcc/archive/2007/11/01/technical-debt- 2.aspx • Sonar – http://www.sonarsource.org/evaluate-your-technical-debt-with-sonar/ • Pay Down your Technical Debt – www.codinghorror.com/blog/2009/02/paying-down-your-technical-debt.html • SQALE Method For Evaluating Technical Debt – http://www.sqale.org/wp-content/uploads/2012/04/SQALE-3RD-WS-on-MTD.pdf
- 65. Pictures • http://www.flickr.com/photos/49531720@N00/247730111/ • http://www.flickr.com/photos/89306448@N00/2247180420/ • http://www.flickr.com/photos/71962092@N00/2874328851 • http://www.flickr.com/photos/16857236@N03/2429136239 • http://www.flickr.com/photos/tpapi/2765541278/ • http://www.flickr.com/photos/97041449@N00/5261698908/ • http://www.flickr.com/photos/7389424@N05/2351559480/ • http://www.flickr.com/photos/24293932@N00/1144691293/ • http://www.flickr.com/photos/17454738@N00/2245445147/
- 66. Pictures • http://www.flickr.com/photos/25196025@N00/381877979/ • http://www.flickr.com/photos/25507200@N07/3120849218/ • http://www.flickr.com/photos/39516732@N08/4666623572/ • http://www.flickr.com/photos/64211362@N02/6338814898/ • http://www.flickr.com/photos/66622362@N00/3353570653/ • http://www.flickr.com/photos/23327787@N08/3027534098/ • http://www.flickr.com/photos/37815348@N00/5398908333/ • http://www.flickr.com/photos/51035555243@N01/155589939/
Editor's Notes
- Definition of done Debt conversion Probability of occurance Heat maps
- Agenda
- About me
- What’s going on? Start a little slow Quickly pick up - A lot of features are getting developed Things start to slow down Can’t seem to recover These are signs of bad code and technical debt
- Adding new features or simple changes take forever Picture: http://www.flickr.com/photos/49531720@N00/247730111/
- Delivery dates are missed Picture: http://www.flickr.com/photos/89306448@N00/2247180420/
- Bugs increase Picture: http://www.flickr.com/photos/71962092@N00/2874328851/
- Team is frustrated Morale is low Picture: http://www.flickr.com/photos/16857236@N03/2429136239/
- Customer is unhappy
- Uncle Bob provides the following attributes of bad code: Rigidity – difficult to change. A change causes a cascade of subsequent changes in dependent modules. Result -> manager fear to fix non critical problems because no reliability on impact and how long it will take. This lead to rigidity.
- Fragility – closely related to rigidity. Software tends to break in many places every time it is changed. Even in areas that are conceptually unrelated. As this increases, software becomes impossible to maintain because every fix introduces 5 new bugs. This leads to distrust and a loss of credibility.
- Developers end up playing whack-a-bug. Picture: http://www.flickr.com/photos/tpapi/2765541278/
- Immobility – inability to reuse software from other parts of the system. Need module similar to another one already written, however, module has too much baggage. It is hard and risky to separate desirable part from undesirable part so decide to rewrite instead of reuse. Picture: http://www.flickr.com/photos/97041449@N00/5261698908/
- Viscosity – design (when faced with a change, take easy way(hack) or hard way) Environment- If compile time is long, make changes that do not force large recompile If check-in takes a long time, make as few check ins as possible If deployment is difficult, make changes in database. End up applying the easiest change to avoid environment issues even if change is in the wrong place.
- Reasons for bad code: Excuses about deadlines – no time to test, no time for design change
- Broken Window Theory: Police in NYC conducted a study 1. Parked a car in a bad neighborhood and observed 2. Nothing happened for a couple of days 3. Police broke small rear window and observed 4. Within 4 hours, car was stripped Picture: http://www.flickr.com/photos/7389424@N05/2351559480/
- http://www.flickr.com/photos/24293932@N00/1144691293/
- Broken window theory also applies to code. Start out and things are clean Once you introduce one sign of bad code, things quickly get out of hand As developers, we’ll say, well we are already doing things wrong here, so it does not matter if we do it wrong here as well. Soon things spread and get out of control. Picture: http://www.flickr.com/photos/17454738@N00/2245445147/
- Over architecting: Attempt to envision all possible future scenarios Create unnecessary layers of complexity to support things that are not really requirements Tempted to design solutions to problems in advance of their needs Program goes in a different direction and code is written without benefit Code doesn’t really fit need because written before problem well understood Overdesign takes extra time and produces code that is either unused or doesn’t fit the needs of the project
- Sometimes design is bad and instead of stopping to fix we continue building on top of it and create an even bigger mess. Picture: http://www.flickr.com/photos/25196025@N00/381877979/
- Poor skills or lack of proper training is also a major contributor http://www.flickr.com/photos/25507200@N07/3120849218/
- Technical debt by Ward Cunningham
- Principal: Cost due to amount of time we need to code it the right way Interest: Cost due to amount of extra time we spend to work with something not coded the right way If it costs the same (no interest), it is just differed work and not technical debt
- Jim Highsmith about technical debt in terms of impact on Cost of Change
- We trade on quality daily. Have option to buy quality watch, but decide to buy cheaper one with out knowing what is inside. No visibility into internal quality. Picture: http://www.flickr.com/photos/39516732@N08/4666623572/
- Looks the same, and cheaper! Picture: http://www.flickr.com/photos/64211362@N02/6338814898/
- Martin Fowler about design stamina hypothesis: Some benefit by ignoring good design Soon productivity flattens out Good design route starts out slower, but soon becomes almost linear Intersection is design pay-off line If above the line, no longer makes sense to ignore good design decision Trick to know how far from line. Martin estimates in most cases this is in magnitude of weeks not months or years
- Is all technical debt bad? Quick and dirty or clean? Which one would you choose? It depends…
- Not all technical debt is bad Loans to buy a house or a car are generally considered good loans We want to be responsible in our decisions
- Martin Fowler introduces the technical debt quadrant: 1. Inadvertent/Reckless: Don’t know that they are doing something wrong 2. Deliberate/Reckless: Know it wrong and continue to do it 3. Deliberate/Prudent: Make conscious decision to take on debt due to specific business need 4. Inadvertent/Prudent: With the best of intentions, things did not go well (rare) Most people are in quadrant 1 and 2 and we need to move them to 3. There are good reasons to take on debt like a shorter time to market, preserving startup capital, delaying development expenses
- Types of debt Unintentional debt: careless mistakes, and honest mistakes (bad judgment) – never a good idea Intentional debt – short term (focused and unfocused), long term Only short focused and long term good debt
- Flower shop website must be live 2 weeks before valentines day Otherwise business will go under Must take on debt to meet date Will spend x weeks paying back debt while preparing for Mother’s day
- Make prudent and deliberate decision on debt Have plan to service debt Make our payments Avoid late payment notices
- We don’t want to take on debt when we are behind schedule. This causes a vicious circle
- If debt not paid back, business shuts down or application will require re-write Picture: www.flickr.com/photos/66622362@N00/3353570653/
- Step 1 - Register our debt Add shortcuts decisions to technical debt backlog Include estimated hours to do it the right way Add date of estimate because only valid if paid off immediately
- We want to try to determine Cost of clean way Cost of dirty way Savings now Impact (interest rate) Cost of fix later Beware of Broken Windows Avoid debt that is untrackable Avoid low ROI
- We want to try to determine Cost of clean way Cost of dirty way Savings now Impact (interest rate) Cost of fix later Beware of Broken Windows Avoid debt that is untrackable Avoid low ROI
- Step 2 – Evaluate existing code Inspect and assess level of debt and also register it
- Sonarqube
- Various metrics with drill down capabilities
- Various metrics with drill down capabilities
- Step 3 - monetize the debt 1. Estimate the amount of hours it takes to fix each type of deficit 2. Multiply by rate/hr 3. Multiply by the total number of items
- Technical debt plugin in Sonar calculates it this way
- Another plugin is the SQALE metric (Software Quality Assessment based on Life Cycle Expectations) that breaks things along characteristics (non functional requirements), sub characteristics, and source code requirements Changeability -> loose coupling or related changeability -> code duplication Maintainability -> understandability -> redundant throws or simplifying Boolean expressions, or unused variables Security -> Errors -> preserving track trace, sql injection, or system.exit Reliability - > unit test or exception handling -> unit test coverage, illegal throws Testability -> unit level testability -> cyclomatic complexity Efficiency -> memory usage -> correct data types or static final or string buffering Portability -> software related portability -> imports, or locale The method defines 4 key concepts: 1. The Quality Model: The SQALE Quality Model is used for formulating and organizing the non-functional requirements that relate to code quality. It is organized in three hierarchical levels. The first level is composed of characteristics, the second of sub-characteristics. The third level is composed of requirements that relate to the source code’s internal attributes. These requirements usually depend on the software context and language. 2. The Analysis Model: The SQALE Analysis Model contains on the one hand the rules that are used for normalizing the measures and the violations relating to the code, and on the other hand the rules for aggregating the normalized values. 3. The Indices: All indices represent costs. One represents the Technical Debt of the application. Others are used to analyze that debt. 4. The Indicators: The method defines three summarized indicators. They have been defined to provide a visual representation of the Technical Debt.
- Example dashboard from sonarqube
- Example dashboard from sonarqube
- Costs can be customized for own environment Future work: Prioritization, based on impact. Try to link cost vs. impact on business. Example, fixing non-initialized variable is low cost, however, it can cause large damage
- With dollar amount associated with technical debt, we can engage upper management with a language they understand
- Agile Triangle by Jim Highsmith
- Technical debt triangle by Israel Gat
- Step 4 – Establish debt limit We can now have a conversation with the business in $$$ and talk about technical debt and the impact on ROI. Today about 70% of an organizations IT budget is spent on maintenance instead of new development. What is a good amount of debt? It depends on appetite for risk. Tension between business and tech Biz optimistic about benefit and the cost of carrying the debt Tech pessimistic about benefit and costs Key is to manage and monitor it. If reach certain threshold, then make major payments to get back on track and keep debt under control.
- Credit rating: based on teams ability to avoid unintentional technical debt Acquired debt: merger with other companies, decisions made by others 90 day same as cash: if you fix it quickly there will be no interest payment Low monthly payment: web vs. embedded software – ability to payoff debt depends on type of software Retiring debt: Once we retire system, there is no more debt. This is hard with financial debt Deficit programming Technical Inflation: Failing behind in versions to the point where the code is no longer compatible with the compiler The key is that technical debt enables a conversation between the technical folks and the business folks to make informative decision on code quality
- Step 5 – Pay down debt Where do we start? Use sonar to help you narrow down on problem areas
- Tool gives us information on where to focus
- Step 5a – Pay high interest debt 1st Don’t quickly jump to fix most complex class. There are long term debt and short term debt. 1st reduce short term debt which is like credit card debt. Short term debt has highest interest rate. Tackle 1st the classes that are constantly changing the most. The more changes we make the more interest we are paying. Consider high minimum payment & required on going payment. A class with no test coverage at all, or one that is very complex, but is never or rarely modified, needs to be left on the side for now and is considered as a low interest debt. Focus on the high interest 1st. Picture: http://www.flickr.com/photos/23327787@N08/3027534098/ Picture: http://www.flickr.com/photos/37815348@N00/5398908333/
- Step 5b – Pay down debt based on team skills Does the team know how to write tests to increase code coverage? Does the team know how to reduce complexity? If not, come up with a training and coaching plan. Does the team understand the rules they are violating? If so start their. Do they know how to remove duplications? It is going to depend, but the key is to put a plan and execute on it. Also, don’t try to many things at once. This can be overwhelming. Reduce debt while fixing bugs Do not take on additional debt for new code Picture: http://www.flickr.com/photos/51035555243@N01/155589939/
- Approach: Immediately after a release 10% of each iteration Above certain threshold Rotate individual
- Research for this presentation based mainly on the work of Robert Martin, Steve McConnell, Israel Gat, Martin Fowler