Managing Technical Debt


           Managing Technical Debt


Fadi Stephan
Fadi.Stephan@excella.com
@FadiStephan
Excella.com
AgileJourneyman.com
About Fadi Stephan
                     • 15+ years of experience in
                       software development
                     • Focused on Agile since
                       2006
                     • Consultant with Excella
                     • Founder of the DC
                       Software Craftsmanship
                       User Group
                     • Organizer of the DC Scrum
                       User Group
What’s Going On?
45
40
35
30
25
20                                                      Velocity

15
10
5
0
     1   2   3   4   5   6   7   8   9   10   11   12
Rigidity
Immobility
Viscosity
Deadline
Broken Window
Over Architecting
Bad Design
Poor Skills
Technical Debt
“Shipping first time code is like going
into debt. A little debt speeds
development so long as it is paid back
promptly with a rewrite... The danger
occurs when the debt is not repaid.
Every minute spent on not-quite-right
code counts as interest on that debt.”
                         - Ward Cunningham
Technical Debt Metaphor

“Neglecting the design is like borrowing money”

“Developing slower because of this debt is like paying
interest on the loan”


“Refactoring, it's like paying off the principal debt”


“Every minute spent on not-quite-right code counts as
interest on that debt”
Quick and dirty design results in


                      Principal

                      Interest


                      Technical Debt
Trading for Quality
Design Stamina Hypothesis




          martinfowler.com/bliki/DesignStaminaHypothesis.html
Which one will you choose?



     1. Quick and Dirty

     2. Clean
Managing Technical Debt
Given the option between a clean or dirty approach, which one will you choose?

1. Team, this past sprint our velocity was down due to a lot of time spent on UI
issues. Let’s not worry about compatibility with other browsers and just write
enough code to make the application work in IE 8. Don’t worry if the code breaks
things on the other browsers. Our corporate policy is for us to support IE 8 only.

2. Team, we are behind schedule. We need to catch up quickly. Let’s skip unit
testing for now and just get the code working. We’ll come back and write the tests
after we release. The business is counting on us!

3. I think it now makes more sense to have the EMAIL field is the EMPLOYEE table
instead of the ATTRIBUTE table. We don’t have time to change the code, but we
can create a trigger for now to keep the two database tables in sync and we’ll
refactor the code after we release.

4. This story has a bunch of rules that need to be validated in the business layer. It
will probably take 40 hours to do. However, I can write the same logic in the UI
layer a lot quicker. I think I can wrap this up in about 8 hours. I’ll be able to move
the code to the business layer in a future sprint when I have more time.
Case 1
Team, this past sprint our velocity was down due
to a lot of time spent on UI issues. Let’s not
worry about compatibility with other browsers
and just write enough code to make the
application work in IE 8. Don’t worry if the code
breaks things on the other browsers. Our
corporate policy is for us to support IE 8 only.
Case 2
Team, we are behind schedule. We need to
catch up quickly. Let’s skip unit testing for now
and just get the code working. We’ll come back
and write the tests after we release. The
business is counting on us!
Case 3
I think it now makes more sense to have the
EMAIL field is the EMPLOYEE table instead of the
ATTRIBUTE table. We don’t have time to change
the code, but we can create a trigger for now to
keep the two database tables in sync and we’ll
refactor the code after we release.
Case 4
This story has a bunch of rules that need to be
validated in the business layer. It will probably
take 40 hours to do. However, I can write the
same logic in the UI layer a lot quicker. I think I
can wrap this up in about 8 hours. I’ll be able to
move the code to the business layer in a future
sprint when I have more time.
Home or car loan
Technical Debt Quadrant




          martinfowler.com/bliki/TechnicalDebtQuadrant.html
Types of Debt
• Unintentional
• Intentional
  – Short term & focused
  – Short term & unfocused
  – Long term
• Only short term focused debt & long term
  debt are “good” debt


           forums.construx.com/blogs/stevemcc/archive/2007/11/01/technical-debt-2.aspx
theagileexecutive.com/category/technical-debt
Technical Debt Management Plan
Register the Debt
Date: 2/10/2012                   Estimate: 3

 As I prudent developer,
 I am deliberately taking on technical debt by
 …
 …
 so that…
                                  Impact: M
Estimate: 8

As I prudent developer,
I want to refactor ….
…
…
so that I can repay the technical debt
Technical Debt Backlog
Story            Dirty      Clean      On     Date
                 Estimate   Estimate   Going
                                       Impact
…                3          8          H      2/5/2012


…                1          5          M      2/10/2012


…                3          13         L      2/11/2012
Evaluate Code Base


Complexity
Code Coverage
Duplication
Rule Violations
Design
Monetize the Debt

Technical Debt =
 #items * #hours/item * $/hr
Technical Debt Plugin




Debt(in man days) = cost_to_fix_duplications
                   + cost_to_fix_violations
                   + cost_to_comment_public_API
                   + cost_to_fix_uncovered_complexity
                   + cost_to_bring_complexity_below_threshold
                   + cost_to_cut_cycles_at_package_level
SQALE

        Changeability
        Maintainability
        Security
        Reliability
        Testability
        Efficiency
        Portability
sqale.org/wp-content/uploads/2012/04/SQALE-3RD-WS-on-MTD.pdf
Sample Remediation Functions
Requirement          Remediation Details           Remediation
                                                   Function

No commented out     Remove                        1 min/occurrence
blocks

At least 70% code    Write tests                   20 min/per
coverage                                           uncovered line

Code overrides both Write code and tests           1 hr/occurrence
equals and hashcode



                    sqale.org/wp-content/uploads/2012/04/SQALE-3RD-WS-on-MTD.pdf
SQALE Pyramid




 sqale.org/wp-content/uploads/2012/04/SQALE-3RD-WS-on-MTD.pdf
History
Cost = 2,000,000
Profit=10,000,000
Debt =3,000,000

ROI = (10M – 2M)/ 2M = 400%




                       theagileexecutive.com/category/technical-debt/
theagileexecutive.com/tag/the-agile-triangle/
10,000,000




3,000,000                                            2,000,000

       ROI = (10M – (2M + 3M))/ 5M = 100%

                          theagileexecutive.com/tag/the-agile-triangle/
How much debt is too much debt?
Metaphor
• Think of 3 more examples of ways to use the
  technical debt metaphor
  – Analogy 1:
  – Analogy 2:
  – Analogy 3:
• Do you think the technical debt metaphor
  works well?
• If not, why?
Paying Down The Debt
Pay debt with high interest rate 1st
Approach
• Have a technical debt reduction sprint
  immediately after a release
• Have a technical debt reduction sprint once
  we reach a certain limit
• Rotate dedicated members to work on
  reducing technical debt
• Dedicate 10% of each sprint to reducing
  technical debt
• Reduce technical debt by story
Summary


Managing technical debt requires that we make
prudent and deliberate decision on
design & quality
Summary


Provide transparency by
  1. Registering any new debt
  2. Assessing existing debt
Summary


Inspect by
   1. Monetizing the debt
   2. Establishing a debt limit
   3. Monitor trends
Summary


Adapt by
  1. Paying down the debt focusing on high
     interest rate 1st.
  2. Starting with what you know. Train for the
     rest
  3. Continuously monitor the debt
AgileJourneyman.com
@FadiStephan
Acknowledgement




Robert Martin   Steve McConnell   Israel Gat   Martin Fowler
References
• Design Principles and Design Patterns - Robert Martin
• Design Stamina Hypothesis -
  martinfowler.com/bliki/DesignStaminaHypothesis.html
• Technical Debt Quadrant -
  martinfowler.com/bliki/TechnicalDebtQuadrant.html
• The Agile Triangle –
  theagileexecutive.com/tag/the-agile-triangle/
• Technical Debt Assessment and Reduction –
  theagileexecutive.com/category/technical-debt/
• Technical Debt, Cutter IT Journal October 2010 -
  www.cutter.com
References
• Technical Debt A Perspective for Manager –
  www.infoq.com/articles/technical-debt-levison

• Managing Technical Debt -
  blogs.versionone.com/agile_management/2011/07/11/managing-technical-debt/

• What Testers Can Do About Technical Debt -
  www.stickyminds.com/sitewide.asp?ObjectId=3629

• Climb Out of Technical Debt –
  www.ayeconference.com/climboutoftechnicaldebt/

• Don't Live with Broken Windows –
  www.artima.com/intv/fixit.html
References
• Technical Debt -
  forums.construx.com/blogs/stevemcc/archive/2007/11/01/technical-debt-
  2.aspx

• Sonar –
  http://www.sonarsource.org/evaluate-your-technical-debt-with-sonar/

• Pay Down your Technical Debt –
  www.codinghorror.com/blog/2009/02/paying-down-your-technical-debt.html

• SQALE Method For Evaluating Technical Debt –
  http://www.sqale.org/wp-content/uploads/2012/04/SQALE-3RD-WS-on-MTD.pdf
Pictures
•   http://www.flickr.com/photos/49531720@N00/247730111/
•   http://www.flickr.com/photos/89306448@N00/2247180420/
•   http://www.flickr.com/photos/71962092@N00/2874328851
•   http://www.flickr.com/photos/16857236@N03/2429136239
•   http://www.flickr.com/photos/tpapi/2765541278/
•   http://www.flickr.com/photos/97041449@N00/5261698908/
•   http://www.flickr.com/photos/7389424@N05/2351559480/
•   http://www.flickr.com/photos/24293932@N00/1144691293/
•   http://www.flickr.com/photos/17454738@N00/2245445147/
Pictures
•   http://www.flickr.com/photos/25196025@N00/381877979/
•   http://www.flickr.com/photos/25507200@N07/3120849218/
•   http://www.flickr.com/photos/39516732@N08/4666623572/
•   http://www.flickr.com/photos/64211362@N02/6338814898/
•   http://www.flickr.com/photos/66622362@N00/3353570653/
•   http://www.flickr.com/photos/23327787@N08/3027534098/
•   http://www.flickr.com/photos/37815348@N00/5398908333/
•   http://www.flickr.com/photos/51035555243@N01/155589939/

Managing technical debt

  • 1.
    Managing Technical Debt Managing Technical Debt Fadi Stephan Fadi.Stephan@excella.com @FadiStephan Excella.com AgileJourneyman.com
  • 3.
    About Fadi Stephan • 15+ years of experience in software development • Focused on Agile since 2006 • Consultant with Excella • Founder of the DC Software Craftsmanship User Group • Organizer of the DC Scrum User Group
  • 4.
    What’s Going On? 45 40 35 30 25 20 Velocity 15 10 5 0 1 2 3 4 5 6 7 8 9 10 11 12
  • 10.
  • 13.
  • 14.
  • 15.
  • 16.
  • 19.
  • 20.
  • 21.
  • 22.
    Technical Debt “Shipping firsttime code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite... The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt.” - Ward Cunningham
  • 23.
    Technical Debt Metaphor “Neglectingthe design is like borrowing money” “Developing slower because of this debt is like paying interest on the loan” “Refactoring, it's like paying off the principal debt” “Every minute spent on not-quite-right code counts as interest on that debt”
  • 24.
    Quick and dirtydesign results in Principal Interest Technical Debt
  • 27.
  • 28.
    Design Stamina Hypothesis martinfowler.com/bliki/DesignStaminaHypothesis.html
  • 29.
    Which one willyou choose? 1. Quick and Dirty 2. Clean
  • 30.
    Managing Technical Debt Giventhe option between a clean or dirty approach, which one will you choose? 1. Team, this past sprint our velocity was down due to a lot of time spent on UI issues. Let’s not worry about compatibility with other browsers and just write enough code to make the application work in IE 8. Don’t worry if the code breaks things on the other browsers. Our corporate policy is for us to support IE 8 only. 2. Team, we are behind schedule. We need to catch up quickly. Let’s skip unit testing for now and just get the code working. We’ll come back and write the tests after we release. The business is counting on us! 3. I think it now makes more sense to have the EMAIL field is the EMPLOYEE table instead of the ATTRIBUTE table. We don’t have time to change the code, but we can create a trigger for now to keep the two database tables in sync and we’ll refactor the code after we release. 4. This story has a bunch of rules that need to be validated in the business layer. It will probably take 40 hours to do. However, I can write the same logic in the UI layer a lot quicker. I think I can wrap this up in about 8 hours. I’ll be able to move the code to the business layer in a future sprint when I have more time.
  • 31.
    Case 1 Team, thispast sprint our velocity was down due to a lot of time spent on UI issues. Let’s not worry about compatibility with other browsers and just write enough code to make the application work in IE 8. Don’t worry if the code breaks things on the other browsers. Our corporate policy is for us to support IE 8 only.
  • 32.
    Case 2 Team, weare behind schedule. We need to catch up quickly. Let’s skip unit testing for now and just get the code working. We’ll come back and write the tests after we release. The business is counting on us!
  • 33.
    Case 3 I thinkit now makes more sense to have the EMAIL field is the EMPLOYEE table instead of the ATTRIBUTE table. We don’t have time to change the code, but we can create a trigger for now to keep the two database tables in sync and we’ll refactor the code after we release.
  • 34.
    Case 4 This storyhas a bunch of rules that need to be validated in the business layer. It will probably take 40 hours to do. However, I can write the same logic in the UI layer a lot quicker. I think I can wrap this up in about 8 hours. I’ll be able to move the code to the business layer in a future sprint when I have more time.
  • 35.
  • 36.
    Technical Debt Quadrant martinfowler.com/bliki/TechnicalDebtQuadrant.html
  • 37.
    Types of Debt •Unintentional • Intentional – Short term & focused – Short term & unfocused – Long term • Only short term focused debt & long term debt are “good” debt forums.construx.com/blogs/stevemcc/archive/2007/11/01/technical-debt-2.aspx
  • 40.
  • 42.
  • 43.
  • 44.
    Date: 2/10/2012 Estimate: 3 As I prudent developer, I am deliberately taking on technical debt by … … so that… Impact: M
  • 45.
    Estimate: 8 As Iprudent developer, I want to refactor …. … … so that I can repay the technical debt
  • 46.
    Technical Debt Backlog Story Dirty Clean On Date Estimate Estimate Going Impact … 3 8 H 2/5/2012 … 1 5 M 2/10/2012 … 3 13 L 2/11/2012
  • 47.
    Evaluate Code Base Complexity CodeCoverage Duplication Rule Violations Design
  • 51.
    Monetize the Debt TechnicalDebt = #items * #hours/item * $/hr
  • 52.
    Technical Debt Plugin Debt(inman days) = cost_to_fix_duplications + cost_to_fix_violations + cost_to_comment_public_API + cost_to_fix_uncovered_complexity + cost_to_bring_complexity_below_threshold + cost_to_cut_cycles_at_package_level
  • 53.
    SQALE Changeability Maintainability Security Reliability Testability Efficiency Portability
  • 55.
  • 56.
    Sample Remediation Functions Requirement Remediation Details Remediation Function No commented out Remove 1 min/occurrence blocks At least 70% code Write tests 20 min/per coverage uncovered line Code overrides both Write code and tests 1 hr/occurrence equals and hashcode sqale.org/wp-content/uploads/2012/04/SQALE-3RD-WS-on-MTD.pdf
  • 57.
  • 58.
  • 59.
    Cost = 2,000,000 Profit=10,000,000 Debt=3,000,000 ROI = (10M – 2M)/ 2M = 400% theagileexecutive.com/category/technical-debt/
  • 60.
  • 61.
    10,000,000 3,000,000 2,000,000 ROI = (10M – (2M + 3M))/ 5M = 100% theagileexecutive.com/tag/the-agile-triangle/
  • 62.
    How much debtis too much debt?
  • 63.
    Metaphor • Think of3 more examples of ways to use the technical debt metaphor – Analogy 1: – Analogy 2: – Analogy 3: • Do you think the technical debt metaphor works well? • If not, why?
  • 64.
  • 68.
    Pay debt withhigh interest rate 1st
  • 70.
    Approach • Have atechnical debt reduction sprint immediately after a release • Have a technical debt reduction sprint once we reach a certain limit • Rotate dedicated members to work on reducing technical debt • Dedicate 10% of each sprint to reducing technical debt • Reduce technical debt by story
  • 71.
    Summary Managing technical debtrequires that we make prudent and deliberate decision on design & quality
  • 72.
    Summary Provide transparency by 1. Registering any new debt 2. Assessing existing debt
  • 73.
    Summary Inspect by 1. Monetizing the debt 2. Establishing a debt limit 3. Monitor trends
  • 74.
    Summary Adapt by 1. Paying down the debt focusing on high interest rate 1st. 2. Starting with what you know. Train for the rest 3. Continuously monitor the debt
  • 75.
  • 76.
    Acknowledgement Robert Martin Steve McConnell Israel Gat Martin Fowler
  • 77.
    References • Design Principlesand Design Patterns - Robert Martin • Design Stamina Hypothesis - martinfowler.com/bliki/DesignStaminaHypothesis.html • Technical Debt Quadrant - martinfowler.com/bliki/TechnicalDebtQuadrant.html • The Agile Triangle – theagileexecutive.com/tag/the-agile-triangle/ • Technical Debt Assessment and Reduction – theagileexecutive.com/category/technical-debt/ • Technical Debt, Cutter IT Journal October 2010 - www.cutter.com
  • 78.
    References • Technical DebtA Perspective for Manager – www.infoq.com/articles/technical-debt-levison • Managing Technical Debt - blogs.versionone.com/agile_management/2011/07/11/managing-technical-debt/ • What Testers Can Do About Technical Debt - www.stickyminds.com/sitewide.asp?ObjectId=3629 • Climb Out of Technical Debt – www.ayeconference.com/climboutoftechnicaldebt/ • Don't Live with Broken Windows – www.artima.com/intv/fixit.html
  • 79.
    References • Technical Debt- forums.construx.com/blogs/stevemcc/archive/2007/11/01/technical-debt- 2.aspx • Sonar – http://www.sonarsource.org/evaluate-your-technical-debt-with-sonar/ • Pay Down your Technical Debt – www.codinghorror.com/blog/2009/02/paying-down-your-technical-debt.html • SQALE Method For Evaluating Technical Debt – http://www.sqale.org/wp-content/uploads/2012/04/SQALE-3RD-WS-on-MTD.pdf
  • 80.
    Pictures • http://www.flickr.com/photos/49531720@N00/247730111/ • http://www.flickr.com/photos/89306448@N00/2247180420/ • http://www.flickr.com/photos/71962092@N00/2874328851 • http://www.flickr.com/photos/16857236@N03/2429136239 • http://www.flickr.com/photos/tpapi/2765541278/ • http://www.flickr.com/photos/97041449@N00/5261698908/ • http://www.flickr.com/photos/7389424@N05/2351559480/ • http://www.flickr.com/photos/24293932@N00/1144691293/ • http://www.flickr.com/photos/17454738@N00/2245445147/
  • 81.
    Pictures • http://www.flickr.com/photos/25196025@N00/381877979/ • http://www.flickr.com/photos/25507200@N07/3120849218/ • http://www.flickr.com/photos/39516732@N08/4666623572/ • http://www.flickr.com/photos/64211362@N02/6338814898/ • http://www.flickr.com/photos/66622362@N00/3353570653/ • http://www.flickr.com/photos/23327787@N08/3027534098/ • http://www.flickr.com/photos/37815348@N00/5398908333/ • http://www.flickr.com/photos/51035555243@N01/155589939/