The document discusses managing software debt as systems age. It covers different types of software debt including technical debt, quality debt, configuration management debt, design debt, and platform experience debt. The document provides examples of how software debt can creep into systems over time if not properly managed through frequent feedback mechanisms and refactoring code as needed.

1. UW Agile CP202 Class 3:
Managing Software Debt
Continued Delivery of High Value as Systems Age
Chris Sterling
Technology Consultant / Agile Coach /
Certified Scrum Trainer
Sterling Barton, LLC
Web: www.SterlingBarton.com
Email: chris@sterlingbarton.com
Blog: www.GettingAgile.com
Follow Me on Twitter: @csterwa
Hash Tag for Presentation: #swdebt
Monday, May 10, 2010 1

2. Topics Being Covered
Problems Found with Aging Software
Software Debt Explained
Technical Debt
Quality Debt
Configuration Management Debt
Design Debt
Platform Experience Debt
The Wrap Up
A Story of What is Possible
Monday, May 10, 2010 2

3. Problems Found with Aging Software
Software gets difficult to add features to as it ages
Business expectations do not lessen as software ages
Software must remain maintainable and changeable to meet needs
of business over time
Monday, May 10, 2010 3

4. Lack of emphasis on software
quality attributes contributes to
decay
Monday, May 10, 2010 4

5. Like-to-Like Migrations
“It will be easy since we worked on the original version” - although
we understand the domain we will be fighting with new features,
technology, tools, and processes
“We don’t have any other options” - Refactoring and test
automation are potential alternatives to like-to-like migrations.
Monday, May 10, 2010 5

6. Limited Platform Expertise
Risk and costs increase as expertise becomes more limited for aging
software platforms.
Monday, May 10, 2010 6

7. Costs for Release Stabilization
500000
375000
250000
125000
Release 6
Release 5
Release 4
0 Release 3
Release 2
Release 1
Cost of Fixing Defects Cost for Feature Dev
Monday, May 10, 2010 7

8. Regression Costs - Manual vs.
Monday, May 10, 2010 8

9. Extreme Specialization
Knowledge and capability to maintain legacy software decays with
time
Costs to maintain rarely used software platforms are higher
Leads to waiting for people in specialized roles to finish their tasks
in support of development effort
Monday, May 10, 2010 9

10. Software Debt
Creeps into software slowly and leaves organizations with liabilities
Monday, May 10, 2010 10

11. Software Debt Creeps In
Shows a relatively new system with little software debt
accrued.
Monday, May 10, 2010 11

12. Software Debt Creeps In
An aging software system slowly incurs significant debt in multiple
functional areas.
Monday, May 10, 2010 12

13. Software Debt Creeps In
An older system has accrued significant debt in all functional areas and
components.
Monday, May 10, 2010 13

14. Managing Software Debt – an
Monday, May 10, 2010 14

15. Managing Software Debt
It is impossible to stop software debt from creeping into our
software
Managing debt in software is based on putting frequent feedback
mechanisms in place for code, quality assurance, configuration
management, design, and organization of people on project team
Feedback mechanisms should be frequent, automated, and easy to
use to support their continued use or modified to new needs
Monday, May 10, 2010 15

16. Types of Software Debt
Technical Debt
Quality Debt
Configuration Management Debt
Design Debt
Platform Experience Debt
Monday, May 10, 2010 16

17. Technical Debt
Issues in software implementation that will impede future development
if left unresolved
Monday, May 10, 2010 17

18. * Ward Cunningham on “Technical Debt”
Technical Debt includes those internal things that you choose not
to do now, but which will impede future development if left
undone. This includes deferred refactoring.
Technical Debt doesn’t include deferred functionality, except
possibly in edge cases where delivered functionality is “good
enough” for the customer, but doesn’t satisfy some standard (e.g., a
UI element that isn’t fully compliant with some UI standard).
* Ward Cunningham - “Technical Debt” - http://c2.com/cgi/wiki?TechnicalDebt
Monday, May 10, 2010 18

19. My Definition of “Technical Debt”
“Technical debt is the decay of component and
inter-component behavior when the application
functionality meets a minimum standard of
satisfaction for the customer.”
Monday, May 10, 2010 19

20. Test-Driven Development
Test-Driven Development (TDD)
The “Flow”
Integrating with Team
Difficulties with TDD
Example TDD Session
Refactoring
Monday, May 10, 2010 20

21. Questions for You...
How many of you are practicing TDD?
What issues are you having currently with TDD?
Monday, May 10, 2010 21

22. TDD - Basic “Flow”
Monday, May 10, 2010 22

23. TDD - Basic “Flow”
Write
Failing
Test
Monday, May 10, 2010 22

24. TDD - Basic “Flow”
Write
Failing
Test
Make
Test
Pass
Monday, May 10, 2010 22

25. TDD - Basic “Flow”
Write
Failing
Test
Refactor
to
Make
Test
Acceptable
Design Pass
Monday, May 10, 2010 22

26. TDD - Basic “Flow”
Write
Failing
Test
Refactor
to
Make
Test
Acceptable
Design Pass
Monday, May 10, 2010 22

27. Steps of TDD “Flow”
1. Write a programmer test
2. Run the programmer test and it should fail (red bar)
3. Write just enough code to make failing test pass
4. Run programmer test successfully (green bar)
5. Refactor code to an acceptable design (green bar)
RISK: Pushing out refactoring to add more code first. Don’t forget to
refactor frequently to an acceptable design.
Monday, May 10, 2010 23

28. Be a User --
“What should the software do next
This question helps you to decide what the next programmer test
should model
Monday, May 10, 2010 24

29. Integrating with Team
Monday, May 10, 2010 25

30. Integrating with Team
Write
Failing
Test
Monday, May 10, 2010 25

31. Integrating with Team
Make
Test
Pass
Write
Failing
Test
Monday, May 10, 2010 25

32. Integrating with Team
Make
Test
Pass
Write
Failing
Integrate
Test with
Team
Monday, May 10, 2010 25

33. Integrating with Team
Make
Test
Pass
Write
Failing
Refactor
to
Integrate
Acceptable
Test with
Team Design
Monday, May 10, 2010 25

34. Integrating with Team
Make
Test
Pass
Write
Failing
Refactor
to
Integrate
Acceptable
Test with
Team Design
Monday, May 10, 2010 25

35. Integrating with Team
Make
Test
Pass
Write
Failing
Refactor
to
Integrate
Acceptable
Test with
Team Design
Monday, May 10, 2010 25

36. Need-Driven Design
Monday, May 10, 2010 26

37. Test-Driven Development
What Causes TDD to be so Difficult to Implement Well?
Monday, May 10, 2010 27

38. Impediments to TDD
This simple technique for developing software, when used in a
disciplined manner, will enable individuals and teams to improve
software quality. The discipline necessary to do TDD is not easily
attainable. Following is a list of environmental issues that lowers
chances of effectively implementing TDD approach.
Monday, May 10, 2010 28

39. Schedule Pressure
Pressure from management and stakeholders to release based on an
unreasonable plan. Integrity of the software is always sacrificed
when the plan is inflexible and unwilling to incorporate reality.
Monday, May 10, 2010 29

40. Lack of Passion
Lack of passion for learning and implementing TDD
effectively on Team. The amount of discipline required
makes passion extremely helpful.
Monday, May 10, 2010 30

41. Insufficient Experience
Insufficient team member ratio who have experience
doing TDD well. When there is little experience on or
coaching for the team success is minimal.
Monday, May 10, 2010 31

42. Legacy Code
If the software’s design is poor or is difficult to test then finding a
starting point could seem impossible. If legacy code is large and
contains no, or only minimal, test coverage then disciplined TDD
will not show visible results for some time.
Monday, May 10, 2010 32

43. Introducing TDD Approach
To successfully adopt TDD, it is important to manage these
environmental issues. This could include managing expectations,
providing the Team support from a coach, and enabling sufficient
learning of tools and techniques while working with an existing
code base.
Monday, May 10, 2010 33

44. Test-Driven Development
Quick Example
Monday, May 10, 2010 34

45. Jitter – Example TDD Session
Fake micro-blogging tool named “Jitter” is made by Seattle-based
fictitious company that focuses on enabling coffee injected folks to
write short messages and have common online messaging
shorthand expanded for easy reading. The user story we are
working on is:
So it is easier to read their kid’s messages, Mothers want to
automatically expand common shorthand notation
The acceptance criteria for this user story are:
LOL, AFAIK, and TTYL are expandable
Expand lower and upper case versions of shorthand
Monday, May 10, 2010 35

46. Expand LOL to “laughing out loud”
public class WhenUsersWantToExpandMessagesThatContainShorthandTest {
@Test
public void shouldExpandLOLToLaughingOutLoud() {
JitterSession session = mock(JitterSession.class);
when(session.getNextMessage()).thenReturn("Expand LOL please");
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(),
equalTo("Expand laughing out loud please"));
}
}
public class MessageExpander {
public String getNextMessage() {
String msg = session.getNextMessage();
return msg.replaceAll("LOL", "laughing out loud");
}
Monday, May 10, 2010 36

47. But wait…what if…?
What if LOL is written in lower case?
What if it is written as “Lol”? Should it be expanded?
What if some variation of LOL is inside a word?
What if characters surrounding LOL are symbols, not letters?
Write these down as upcoming programmer tests as comments so I don’t forget them.
// shouldExpandLOLIfLowerCase
// shouldNotExpandLOLIfMixedCase
// shouldNotExpandLOLIfInsideWord
// shouldExpandIfSurroundingCharactersAreNotLetters
Monday, May 10, 2010 37

48. Expand LOL If Lower Case
@Test
public void shouldExpandLOLIfLowerCase() {
when(session.getNextMessage()).thenReturn("Expand lol please");
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(),
equalTo("Expand laughing out loud please"));
}
This forced use of java.util.regex.Pattern to handle case insensitivity.
public String getNextMessage() {
String msg = session.getNextMessage();
return Pattern.compile("LOL”, Pattern.CASE_INSENSITIVE)
.matcher(msg).replaceAll("laughing out loud");
}
Monday, May 10, 2010 38

49. Don’t Expand “Lol” – Mixed-Case
@Test
public void shouldNotExpandLOLIfMixedCase() {
String msg = "Do not expand Lol please";
when(session.getNextMessage()).thenReturn(msg);
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(), equalTo(msg));
}
This forced me to stop using Pattern.CASE_INSENSITIVE flag in pattern compilation. Only use “LOL” or “lol” for
replacement criteria.
public String getNextMessage() {
String msg = session.getNextMessage();
return Pattern.compile("LOL|lol").matcher(msg)
.replaceAll("laughing out loud");
}
Monday, May 10, 2010 39

50. Don’t Expand “LOL” If Inside Word
@Test
public void shouldNotExpandLOLIfInsideWord() {
String msg = "Do not expand PLOL or LOLP or PLOLP please";
when(session.getNextMessage()).thenReturn(msg);
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(), equalTo(msg));
}
The pattern matching is now modified to use spaces around each variation of valid LOL
shorthand.
return Pattern.compile("sLOLs|slols").matcher(msg)
.replaceAll("laughing out loud");
Monday, May 10, 2010 40

51. Expand “LOL” If Not Inside Word
@Test
public void shouldExpandIfSurroundingCharactersAreNotLetters() {
when(session.getNextMessage()).thenReturn("Expand .lol! please");
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(),
equalTo("Expand .laughing out loud! please"));
}
The final implementation of pattern matching code looks as follows.
return Pattern.compile("bLOLb|blolb").matcher(msg)
.replaceAll("laughing out loud");
Monday, May 10, 2010 41

52. Merciless Refactoring
Refactoring: a disciplined technique for restructuring an existing
body of code, altering its internal structure without changing its
external behavior.*
Merciless: having or showing no [mercy - showing great kindness
toward the distressed]
Relieve your distressed code through kindness and disciplined
restructuring
* From http://www.refactoring.com/
Monday, May 10, 2010 42

53. Where to Start Refactoring?
Does this change directly affect feature I am working on?
Would change add clarity for feature implementation?
Will change add automated tests where there are none?
If “yes” to any question above, ask following question to decide if you
should work on it now:
At first glance, does refactoring look like a large endeavor involving
significant portions of the software’s components?
Monday, May 10, 2010 43

54. When to Stop Refactoring?
Am I refactoring code not directly affected by feature?
Is other code directly affected by feature I am working on that has
not been refactoring sufficiently?
If refactoring is exploding feature estimate given to Customer then
I should bring it up to Team to decide how we should progress
If Team decides that refactoring can be absorbed into current
iteration without affecting delivery on our commitments then
continue refactor
Monday, May 10, 2010 44

55. Automate Testing to Support
Refactoring cannot be done effectively without automated tests
surrounding code
Start by creating automated test which fails
If difficult to create at unit level look at automated acceptance tests
from functional perspective
Over time look for ways to create automated unit tests
Monday, May 10, 2010 45

56. Automated Tests
Application
Monday, May 10, 2010 46

57. Automated Tests
Application
New
Feature
Monday, May 10, 2010 46

58. Automated Tests
Application
New Design Changes
Feature
Monday, May 10, 2010 46

59. Automated Tests
Application
Automated
New Design Changes Regression
Test Run
Feature
Monday, May 10, 2010 46

60. Automated Tests
Application
Automated
New Design Changes Regression
Test Run
Feature
Monday, May 10, 2010 46

61. Principles of Agile Software Quality
The system always runs
No code is written without a failing test
Zero post-iteration bugs
* From “Flawless Iterations” presented by Alex Pukinskis at Agile 2005
Monday, May 10, 2010 47

62. Quality Debt
A lack of quality, either technical or functional, will lessen value per
feature added over time
Monday, May 10, 2010 48

63. Accrual of Quality Debt with Releases
Monday, May 10, 2010 49

64. Break/Fix Only Prolongs the Agony
Monday, May 10, 2010 50

65. Effect of Project Constraints on
Monday, May 10, 2010 51

66. Effect of Project Constraints on
Monday, May 10, 2010 51

67. Acceptance Test-Driven Development
Monday, May 10, 2010 52

68. A Fit Case Study
Cost reduction using Fit for test automation and data conversion
Monday, May 10, 2010 53

69. Manual Regression Testing
Testing was taking 75 person hours during 2 full test runs
consisting of:
Comprehensive manual regression testing
Data conversion and validation
Cost for testing was $17,000 each iteration
Monday, May 10, 2010 54

70. Introducing Fit into Testing Process
After 8 iterations team had introduced healthy amount of Fit
fixtures and automated tests
Reduced 70+ hour test runtime down to 6 hours which now
included:
Fit automated regression testing
Data conversion and validation automated with Fit fixtures
Reduced cost of testing each iteration from $17,000 to $7,000
Monday, May 10, 2010 55

71. Configuration Management Debt
Creating unpredictable and error-prone release management
Monday, May 10, 2010 56

72. Traditional Source Control
Monday, May 10, 2010 57

73. Traditional Source Control
Main Branch
Monday, May 10, 2010 57

74. Traditional Source Control
Code
Complete
Version 1 Integrate for
Branch Version 2
Main Branch
Monday, May 10, 2010 57

75. Traditional Source Control
Code
Complete
Version 1 Integrate for
Branch Version 2
Debt Main Branch
Death March
Monday, May 10, 2010 57

76. Traditional Source Control
Code
Complete
Version 1 Integrate for
Branch Version 2
Debt Main Branch
Death March
{
Debt accrues quickly within stabilization periods
Monday, May 10, 2010 57

77. Flexible Source Control Management
Monday, May 10, 2010 58

78. Flexible Source Control Management
Main Branch
Monday, May 10, 2010 58

79. Flexible Source Control Management
Version 1
Main Branch
Monday, May 10, 2010 58

80. Flexible Source Control Management
Version 1 Version 2
Main Branch
Monday, May 10, 2010 58

81. Flexible Source Control Management
Version 1 Version 2
Main Branch
{
Not Easy! Must have proper infrastructure to do this.
Monday, May 10, 2010 58

82. Continuous Integration
Monday, May 10, 2010 59

83. Scaling to Multiple Integrations
Monday, May 10, 2010 60

84. Design Debt
Design decays when not attended to so design your software
continually
Monday, May 10, 2010 61

85. * Abuse User Stories
Implement Security
for User Information
* From “User Stories Applied” presented by Mike Cohn Agile 2006
Monday, May 10, 2010 62

86. * Abuse User Stories
Implement Security As a Malicious Hacker I
for User Information want to steal credit card
information so that I can
make fraudulent charges
* From “User Stories Applied” presented by Mike Cohn Agile 2006
Monday, May 10, 2010 62

87. Platform Experience Debt
Silos of knowledge and increased specialization will increase cost of
maintenance over time
Monday, May 10, 2010 63

88. Team Configuration Patterns
Virtual Architect Pattern
Integration Team Pattern
Component Shepherd Pattern
Team Architect Pattern
Monday, May 10, 2010 64

89. Virtual Architect Pattern
Enterprise
Planning
Monday, May 10, 2010 65

90. Virtual Architect Pattern
Pros
Share architecture ideas and needs across teams
Based on verbal communication
Cons
Usually singles out special Team Member role
Could lead to top-down architecture decisions
IT may gain extensive influence and begin to run Product
Monday, May 10, 2010 66

91. Integration Team Pattern
All features are
Integrate implemented
Features and integrated
every iteration
Feature
Development
Monday, May 10, 2010 67

92. Integration Team Pattern
Pros
Reduces complexity on Feature Teams
Forces delivery from Integration Team instead of interface and
deployment designs
Cons
Perpetuates specialized roles
Don’t always work on highest value Product Backlog items
Monday, May 10, 2010 68

93. Component Shepherd Pattern
Monday, May 10, 2010 69

94. Component Shepherd Pattern
Pros
Share more knowledge within organization to minimize
platform experience debt
Work on highest value Product Backlog items
Cons
Not always optimal as using individual knowledge
Difficult to learn multiple systems across Teams
Monday, May 10, 2010 70

95. Team Architect Pattern
Monday, May 10, 2010 71

96. Team Architect Pattern
Pros
Team owns architecture decisions
Decisions are made close to implementation concerns
Cons
May not have appropriate experience on Team
Team could get “stuck” on architecture decisions
Monday, May 10, 2010 72

97. What is possible?
High quality can be attained and should enable accelerated feature
delivery at same time.
Monday, May 10, 2010 73

98. A Story: Field Support Application
2000+ users access application each day
Application supports multiple perspectives and workflows from Field
Support Operations to Customer Service
Team of 5 people delivering features on existing Cold Fusion platform
implementation
Migrating to Spring/Hibernate in slices while delivering valuable features
36 2-week Sprints, 33 production releases, and only 1 defect found in
production
So, what was the defect you say? Let me tell you…
Monday, May 10, 2010 74

99. Lets wrap this up...
What should I take away from this?
Monday, May 10, 2010 75

100. Principles for Managing Software Debt
Maintain one list of work
Emphasize quality
Evolve tools and infrastructure continually
Improve system design always
Share knowledge across the organization
And most importantly, get the right people to work on your
software!
Monday, May 10, 2010 76