Understanding and Managing Technical Debt


Published on

Developing a basic understanding of technical debt in software projects, sharing it with management, and managing it with the help of stake holders.

Published in: Economy & Finance, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Objectives for instruction and expected results and/or skills developed from learning.
  • Introductory notes.
  • Relative vocabulary list.
  • Relative vocabulary list.
  • A schedule design for optional periods of time/objectives.
  • Introductory notes.
  • A schedule design for optional periods of time/objectives.
  • A schedule design for optional periods of time/objectives.
  • A schedule design for optional periods of time/objectives.
  • A schedule design for optional periods of time/objectives.
  • A schedule design for optional periods of time/objectives.
  • Example graph/chart.
  • Introductory notes.
  • Example graph/chart.
  • Conclusion to course, lecture, et al.
  • Conclusion to course, lecture, et al.
  • Conclusion to course, lecture, et al.
  • Conclusion to course, lecture, et al.
  • Conclusion to course, lecture, et al.
  • Introductory notes.
  • Example graph/chart.
  • Introductory notes.
  • Example graph/chart.
  • Conclusion to course, lecture, et al.
  • Conclusion to course, lecture, et al.
  • Conclusion to course, lecture, et al.
  • Conclusion to course, lecture, et al.
  • An opportunity for questions and discussions.
  • Introductory notes.
  • A list of procedures and steps, or a lecture slide with media.
  • Understanding and Managing Technical Debt

    1. 1. MANAGING TECHNICAL DEBT Shortcuts that save money and time today can cost you down the road Presented by: Dr. Hassan Amin
    2. 2. About the Instructor
    3. 3. Introduction/Course Description  Introduction  Introductory notes  Introductory notes  Introductory notes
    4. 4. About Technical Debt Ward Cunningham proposed the concept of technical debt in 1992. Technical debt is acquired when engineers take shortcuts that fall short of best practices, such as:  Sneaking around an abstraction because it is too hard (or impossible) to figure how to ―do it right‖  Skipping on documentation (both in the code and external documentation)  Using an obscure or incomplete error message because it’s just too hard to create something more informative
    5. 5. What is Technical Debt      Implementing code using a simple but slow algorithm even though they know that a better algorithm will be needed in production using void* when you really should have created an appropriate union* Skimping on good security practices Not writing unit test cases You know refactoring is required, because  Bad Smells of Code  Bad Smells of Design
    6. 6. About Technical Debt   Technical debt may also be taken on intentionally as a strategy to save time or money Not all technical debt is bad. Using a simple but slow algorithm in a prototype can be exactly the correct path, as long as you know how to update the code before it ships.
    7. 7. About Technical Debt That means:  Allowing time in the schedule  Making sure the issue is tracked so it doesn’t get lost in the shuffle  Knowing when you implement the code that a good algorithm actually does exist that will work in this instance  and trusting that management will support you.
    8. 8. Comparison with Financial Debt
    9. 9. Properties of Financial Debt    Repayment is a must It is paid back with interest Inability to repay means that there is a very high cost, be it declaring bankruptcy, losing your house
    10. 10. Technical debt (Similarities)    You generally do have to plan pay back strategy (i.e., rewrite the code or otherwise fix the problem) ―interest‖ is accrued every time you or anyone else (support-desk workers, future programmers, customers, etc.) working with your system is delayed because of bugs, performance problems, time spent researching what has gone wrong, etc. Failure to fix problems can result in the utter collapse of a system—the customer gives up and goes elsewhere, the system becomes so slow and brittle that it has to be rewritten from scratch, and so on.
    11. 11. Technical Debt (Differences) The person who takes on technical debt isn’t necessarily the one who has to pay it off:  Generally developers do not maintain their own code.  Many time end users have to pay. This gives the initial developer little incentive to do the job right the first time unless such an incentive is explicitly created by Project Manager and Product owner .
    12. 12. Paying Back Technical Debt The cost of paying back technical debt comes in the form of the engineering time it takes to rewrite or refactor the code or otherwise fix the problem.
    13. 13. Technical debt (Differences)    Unlike financial debt, technical debt almost never has to be paid off in its entirety. Most (probably all) production systems have bugs/shortcomings that don’t have significant impact on the usability or long-term maintainability of the final system. The problem is that it can be difficult to know in advance which debts will ultimately have the highest cost.
    14. 14. Variables in Engineering  Variables are: Time  Functionality  Resources  Debt      Commonly debt is the free variable in the equation. Debt can seem ―free‖ at first, but technical debt tends to build on itself. Debt involves interest in the form of increased effort to maintain and extend the system. So as you take on debt, it gets harder and takes longer to do maintenance and extension.
    15. 15. Realms of Technical Debt Technical debt has much wider implications:  Avoiding a disk-array upgrade is a tradeoff between technical debt and financial costs.  Failure to consider power and cooling requirements when adding new, hotter equipment to a machine room is a debt.  Failure to automate a simple-but-tedious manual process is a debt.  Systems administrators who have neither documented the systems they support nor trained co-workers before going on vacation are another example.  Having parts of systems that are understood by only one person is another form of debt.
    16. 16. Managing your Debt
    17. 17. Introduction Technical debt is inevitable. The reality is that requirements always change. You can manage technical debt in the following ways:  Agile programming  Documentation  System security  Release cycle management
    18. 18. Agile Programming   It is often better to have a working prototype (even though it isn’t complete or perfect) so that you and the customers can start gaining experience with the system. This is the philosophy behind Agile programming, which accepts some technical debt as inevitable but also mandates a ―plan for change‖.
    19. 19. Documentation   Failure to write the documentation (both internal and external) for the initial prototype may be a good tradeoff, but the longer it goes the harder it is to write. Human memory is transient, and if you show most people code they wrote a year ago they will have to study it to remember why they did it that way.
    20. 20. System Security    Some forms of technical debt are so expensive that they should be avoided entirely whenever possible. Security is an area where taking shortcuts can lead to disaster. Always have passwords and other security checks in place.
    21. 21. Release Cycle Management    The modern trend to ―release early and often,‖ especially in the context of web-based services, has made it easier to take on technical debt and still easier to resolve it. When well-managed, taking on debt earlier allows you to release more functionality earlier, allowing immediate feedback from customers, resulting in a product that is more responsive to user needs. If that debt is not paid off promptly, however, it also compounds more quickly, and the system can bog down at a truly frightening rate.
    22. 22. Technical Debt - Originators Not all technical debt is the result of programmer laziness.  Management all too often aids and abets this problem.  Marketing departments often encourage engineering to move on to the next great thing rather than spending the time necessary to consolidate, clean up, and document the existing system.
    23. 23. An Agile Approach to Reduce Technical Debt
    24. 24. Introduction   Agile development methods and principles have lent visibility to technical debt. Different factors influencing the way agile development teams tackle technical debt are: Developers know about technical debt and are aware that it is important to face this problem.  It’s not clear who is responsible for the reduction of technical debt: the team, IT leads, product owner, or the scum master  It’s difficult to plan the work and define a strategy to reduce technical debt because it is not a part of the regular development process which mainly focuses on implementing features. 
    25. 25.     Product Owner often doesn’t understand the need and benefits of reducing technical debt and don’t consider or allow technical projects / stories in their backlog and release plan. Problems and goals regarding technical debt are neither structured nor documented. At best the things to do are quite clear among team members and partly exist in fragmented documentations. But it could also be worst case that topic owners left the company and took all their knowledge and expertise with them.
    26. 26. Here is a pragmatic approach that allows to understand and reduce the existing debt in an agile way (iterative, prioritized, focused).  Step 1: Involve the Product Owner and ―promote‖ him to be the sponsor of technical debt reduction.  Step 2: Inventory and structure known technical debt  Step 3: Agile prioritization and estimation of the work
    27. 27.   Step 4: Analysis of the data and development of the right strategy. Step 5: Integration into the existing development process.
    28. 28. Involve the Product Owner Step 1: Involve the Product Owner and “Promote” him to be the sponsor of technical debt reduction.  Give your product owner the guarantee that growth of the product will stay the most important part – but not just in a short team (Performance) but also in a long term (Health) manner.  Present different options of how to reduce technical debt to him and involve him in discussion to find the best approaches.
    29. 29. Inventory and Structure Known Debt Step 2: Inventory and Structure Known Technical Debt It’s about extracting knowledge about the systems from the heads to develop a common picture of existing technical problems.  Write the names / identifiers of all applications and modules you own on cards.  Pin these cards on a whiteboard.  Extracted to-do’s (to solve existing problems) from all documentation media you use (wiki, jira, confluence, code documentation, paper), write them on post-its and stick them next to the application name it belongs to.
    30. 30. Every team member is responsible to complete, restructure, and correct the board during this period so that you can go on with a round portfolio of the existing debt in the system.
    31. 31. Prioritization & Estimation of Work Step 3: Agile Prioritization and Estimation of the Work  Now a repayment plan is needed.  The goal is to find good hints for defining a strategy to reduce or even remove the existing debt in an effective way.  Start by establishing two abstract units that are somehow semantically based on concrete dimensions
    32. 32.  On Y axis:  TechDollar for the size of repayment for each item, which is in sum the gross amount of debt you have (1, 5, 20, 50, 200).  This number doesn’t relate to the real costs or benefits, it is just an abstract number that allows you to compare items with each other and to generate a feeling about how big the technical debt is.
    33. 33.  On the X axis:  Animal Sizes (later mapped to story points) for the size / effort of each item. (ant = 1, hamster = 3, dog = 8, donkey = 13, elephant = 20).  Again this number does not relate to any real number but allows you to compare items with each other without handling all the complexity behind each item.
    34. 34.   First order the items based on the y axis using tech dollar. Compare items with each other and ask questions like ―Compared to item X how much of your pain would be gone after finishing item Y?
    35. 35.  In the second step, distribute the items of each row (benefit) among the animal sizes. ―How big is item X compared to Y? How long does it take? How complex is it? How uncertain is the field?…‖
    36. 36.   At the end of this exercise, you will have a matrix with ToDos, i.e. technical stories, that are understood by each team member. The positioning in the matrix gives you good hints to develop a strategy for the reduction of technical debt in the next step.
    37. 37. Analysis Step 4: Analysis of the data and development of the right strategy.  First we created stories marked as ―TechnicalDebtItems‖ in JIRA for each task we defined.  For bringing those items into a prioritized order and for drawing the right conclusions, create a chart to visualize how the efforts relate to the payment and vise versa.
    38. 38. This is the result:
    39. 39.  The Y-axis shows the gross debt. The red line visualizes the amount of repayment distributed over the effort. The distribution of efforts on the X-axis on the other hand results from the sum of effort / story points per point category. Earlier, the team only saw the big blockers and the high amount of different tasks without any order or prioritization. Now we have found a way to focus on each next step without losing the whole thing out of focus.
    40. 40. This overview is also a great tool for working with the product owner and other stakeholders because it gives a good transparency regarding technical debt.
    41. 41. Integration Step 5: Integration into the existing development process.  The longest run the team must go is to actually work on the items, reduce the debt and transform a buggy system into a sustainable, solid platform.  You now have a debt burndown which shows the payment over time (calendar weeks).  The team and the product owner can also agree on including bigger technical stories (> 3 points) into the feature planning and development process.
    42. 42. Smaller stories (1-3 points) will be done spontaneously by the developers using their 20% tech and innovation time. The defined portfolio of technical stories is not static.  Developing new stuff always means new debt.  Working with legacy code often results in finding code that is far from being structured well.
    43. 43.     The team may agree on collecting such things on a wiki page when it’s not a bug or critical blocker. Collected items can be reviewed in a quarterly review and if necessary added to the gross debt. Finding and discussing improvements to the process can be part of regular retrospectives the team holds every other week. In the quarterly review, the process and results can be inspected in detail by the team.
    44. 44. Cost of Debt from Various Perspectives
    45. 45. Technical debt affects everyone, but in different ways. Customers  Customers are the ultimate victims.  When the mobile network goes out, when they can’t get their work submitted on time, when their company loses business because they are fighting the software, they pay.
    46. 46.    Help desk Help-desk personnel suffer from almost every aspect of technical debt: poorly designed interfaces, bad or nonexistent documentation, slow algorithms, etc. Customers get more ornery the longer it takes to fix their problem, while the desk often has no direct access to the people who can solve the problem.
    47. 47.  Operations  In a service-oriented environment the operations people spend much of their time paying for decisions that other people made without consulting them.  Sometimes they get to look at the code, sometimes not.  In any case they get to look at the documentation—if it exists.
    48. 48.  Engineers  Doing a quick, highly functional initial implementation makes the programmer look good at the cost of hampering engineers who join the party later.  Such super-stud programmers may not even realize they are taking on the debt if they have limited experience maintaining mature code.
    49. 49.  Marketing  Marketing people can often be the ones pushing hardest for short product development times because they are pressured by sales and the customers to provide new functionality as quickly as possible.  When that new functionality doesn’t work properly in the field, however, they are also the ones on the wrong side of the firing line.
    50. 50.  Management  Bad management often favors a single department to the detriment of others.  If the favored department is marketing or sales, management will be inclined to take on technical debt without understanding the costs.  Management also pays a price.
    51. 51. Conclusion    Technical debt comes from a multitude of causes, often has difficult-to-predict effects, and usually involves a gamble about what will happen in the future. Much of managing technical debt is the same as risk management, and similar techniques can be applied. If technical debt isn’t managed, then it will tend to build up over time, possibly until a crisis results.
    52. 52. Questions/Discussions
    53. 53. Learn More
    54. 54. Thank you