UW Agile CP202 Class 3 Managing Software Debt

1,273 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

UW Agile CP202 Class 3 Managing Software Debt

  1. 1. UW Agile CP202 Class 3: Managing Software Debt Continued Delivery of High Value as Systems Age Chris Sterling Technology Consultant / Agile Coach / Certified Scrum Trainer Sterling Barton, LLC Web: www.SterlingBarton.com Email: chris@sterlingbarton.com Blog: www.GettingAgile.com Follow Me on Twitter: @csterwa Hash Tag for Presentation: #swdebt Monday, May 10, 2010 1
  2. 2. Topics Being Covered Problems Found with Aging Software Software Debt Explained Technical Debt Quality Debt Configuration Management Debt Design Debt Platform Experience Debt The Wrap Up A Story of What is Possible Monday, May 10, 2010 2
  3. 3. Problems Found with Aging Software Software gets difficult to add features to as it ages Business expectations do not lessen as software ages Software must remain maintainable and changeable to meet needs of business over time Monday, May 10, 2010 3
  4. 4. Lack of emphasis on software quality attributes contributes to decay Monday, May 10, 2010 4
  5. 5. Like-to-Like Migrations “It will be easy since we worked on the original version” - although we understand the domain we will be fighting with new features, technology, tools, and processes “We don’t have any other options” - Refactoring and test automation are potential alternatives to like-to-like migrations. Monday, May 10, 2010 5
  6. 6. Limited Platform Expertise Risk and costs increase as expertise becomes more limited for aging software platforms. Monday, May 10, 2010 6
  7. 7. Costs for Release Stabilization 500000 375000 250000 125000 Release 6 Release 5 Release 4 0 Release 3 Release 2 Release 1 Cost of Fixing Defects Cost for Feature Dev Monday, May 10, 2010 7
  8. 8. Regression Costs - Manual vs. Monday, May 10, 2010 8
  9. 9. Extreme Specialization Knowledge and capability to maintain legacy software decays with time Costs to maintain rarely used software platforms are higher Leads to waiting for people in specialized roles to finish their tasks in support of development effort Monday, May 10, 2010 9
  10. 10. Software Debt Creeps into software slowly and leaves organizations with liabilities Monday, May 10, 2010 10
  11. 11. Software Debt Creeps In Shows a relatively new system with little software debt accrued. Monday, May 10, 2010 11
  12. 12. Software Debt Creeps In An aging software system slowly incurs significant debt in multiple functional areas. Monday, May 10, 2010 12
  13. 13. Software Debt Creeps In An older system has accrued significant debt in all functional areas and components. Monday, May 10, 2010 13
  14. 14. Managing Software Debt – an Monday, May 10, 2010 14
  15. 15. Managing Software Debt It is impossible to stop software debt from creeping into our software Managing debt in software is based on putting frequent feedback mechanisms in place for code, quality assurance, configuration management, design, and organization of people on project team Feedback mechanisms should be frequent, automated, and easy to use to support their continued use or modified to new needs Monday, May 10, 2010 15
  16. 16. Types of Software Debt Technical Debt Quality Debt Configuration Management Debt Design Debt Platform Experience Debt Monday, May 10, 2010 16
  17. 17. Technical Debt Issues in software implementation that will impede future development if left unresolved Monday, May 10, 2010 17
  18. 18. * Ward Cunningham on “Technical Debt” Technical Debt includes those internal things that you choose not to do now, but which will impede future development if left undone. This includes deferred refactoring. Technical Debt doesn’t include deferred functionality, except possibly in edge cases where delivered functionality is “good enough” for the customer, but doesn’t satisfy some standard (e.g., a UI element that isn’t fully compliant with some UI standard). * Ward Cunningham - “Technical Debt” - http://c2.com/cgi/wiki?TechnicalDebt Monday, May 10, 2010 18
  19. 19. My Definition of “Technical Debt” “Technical debt is the decay of component and inter-component behavior when the application functionality meets a minimum standard of satisfaction for the customer.” Monday, May 10, 2010 19
  20. 20. Test-Driven Development Test-Driven Development (TDD) The “Flow” Integrating with Team Difficulties with TDD Example TDD Session Refactoring Monday, May 10, 2010 20
  21. 21. Questions for You... How many of you are practicing TDD? What issues are you having currently with TDD? Monday, May 10, 2010 21
  22. 22. TDD - Basic “Flow” Monday, May 10, 2010 22
  23. 23. TDD - Basic “Flow” Write  Failing   Test Monday, May 10, 2010 22
  24. 24. TDD - Basic “Flow” Write  Failing   Test Make  Test   Pass Monday, May 10, 2010 22
  25. 25. TDD - Basic “Flow” Write  Failing   Test Refactor  to   Make  Test   Acceptable   Design Pass Monday, May 10, 2010 22
  26. 26. TDD - Basic “Flow” Write  Failing   Test Refactor  to   Make  Test   Acceptable   Design Pass Monday, May 10, 2010 22
  27. 27. Steps of TDD “Flow” 1. Write a programmer test 2. Run the programmer test and it should fail (red bar) 3. Write just enough code to make failing test pass 4. Run programmer test successfully (green bar) 5. Refactor code to an acceptable design (green bar) RISK: Pushing out refactoring to add more code first. Don’t forget to refactor frequently to an acceptable design. Monday, May 10, 2010 23
  28. 28. Be a User -- “What should the software do next This question helps you to decide what the next programmer test should model Monday, May 10, 2010 24
  29. 29. Integrating with Team Monday, May 10, 2010 25
  30. 30. Integrating with Team Write  Failing   Test Monday, May 10, 2010 25
  31. 31. Integrating with Team Make  Test   Pass Write  Failing   Test Monday, May 10, 2010 25
  32. 32. Integrating with Team Make  Test   Pass Write  Failing   Integrate   Test with  Team Monday, May 10, 2010 25
  33. 33. Integrating with Team Make  Test   Pass Write  Failing   Refactor  to   Integrate   Acceptable   Test with  Team Design Monday, May 10, 2010 25
  34. 34. Integrating with Team Make  Test   Pass Write  Failing   Refactor  to   Integrate   Acceptable   Test with  Team Design Monday, May 10, 2010 25
  35. 35. Integrating with Team Make  Test   Pass Write  Failing   Refactor  to   Integrate   Acceptable   Test with  Team Design Monday, May 10, 2010 25
  36. 36. Need-Driven Design Monday, May 10, 2010 26
  37. 37. Test-Driven Development What Causes TDD to be so Difficult to Implement Well? Monday, May 10, 2010 27
  38. 38. Impediments to TDD This simple technique for developing software, when used in a disciplined manner, will enable individuals and teams to improve software quality. The discipline necessary to do TDD is not easily attainable. Following is a list of environmental issues that lowers chances of effectively implementing TDD approach. Monday, May 10, 2010 28
  39. 39. Schedule Pressure Pressure from management and stakeholders to release based on an unreasonable plan. Integrity of the software is always sacrificed when the plan is inflexible and unwilling to incorporate reality. Monday, May 10, 2010 29
  40. 40. Lack of Passion Lack of passion for learning and implementing TDD effectively on Team. The amount of discipline required makes passion extremely helpful. Monday, May 10, 2010 30
  41. 41. Insufficient Experience Insufficient team member ratio who have experience doing TDD well. When there is little experience on or coaching for the team success is minimal. Monday, May 10, 2010 31
  42. 42. Legacy Code If the software’s design is poor or is difficult to test then finding a starting point could seem impossible. If legacy code is large and contains no, or only minimal, test coverage then disciplined TDD will not show visible results for some time. Monday, May 10, 2010 32
  43. 43. Introducing TDD Approach To successfully adopt TDD, it is important to manage these environmental issues. This could include managing expectations, providing the Team support from a coach, and enabling sufficient learning of tools and techniques while working with an existing code base. Monday, May 10, 2010 33
  44. 44. Test-Driven Development Quick Example Monday, May 10, 2010 34
  45. 45. Jitter – Example TDD Session Fake micro-blogging tool named “Jitter” is made by Seattle-based fictitious company that focuses on enabling coffee injected folks to write short messages and have common online messaging shorthand expanded for easy reading. The user story we are working on is: So it is easier to read their kid’s messages, Mothers want to automatically expand common shorthand notation The acceptance criteria for this user story are: LOL, AFAIK, and TTYL are expandable Expand lower and upper case versions of shorthand Monday, May 10, 2010 35
  46. 46. Expand LOL to “laughing out loud” public class WhenUsersWantToExpandMessagesThatContainShorthandTest { @Test public void shouldExpandLOLToLaughingOutLoud() { JitterSession session = mock(JitterSession.class); when(session.getNextMessage()).thenReturn("Expand LOL please"); MessageExpander expander = new MessageExpander(session); assertThat(expander.getNextMessage(), equalTo("Expand laughing out loud please")); } } public class MessageExpander { public String getNextMessage() { String msg = session.getNextMessage(); return msg.replaceAll("LOL", "laughing out loud"); } Monday, May 10, 2010 36
  47. 47. But wait…what if…? What if LOL is written in lower case? What if it is written as “Lol”? Should it be expanded? What if some variation of LOL is inside a word? What if characters surrounding LOL are symbols, not letters? Write these down as upcoming programmer tests as comments so I don’t forget them. // shouldExpandLOLIfLowerCase // shouldNotExpandLOLIfMixedCase // shouldNotExpandLOLIfInsideWord // shouldExpandIfSurroundingCharactersAreNotLetters Monday, May 10, 2010 37
  48. 48. Expand LOL If Lower Case @Test public void shouldExpandLOLIfLowerCase() { when(session.getNextMessage()).thenReturn("Expand lol please"); MessageExpander expander = new MessageExpander(session); assertThat(expander.getNextMessage(), equalTo("Expand laughing out loud please")); } This forced use of java.util.regex.Pattern to handle case insensitivity. public String getNextMessage() { String msg = session.getNextMessage(); return Pattern.compile("LOL”, Pattern.CASE_INSENSITIVE) .matcher(msg).replaceAll("laughing out loud"); } Monday, May 10, 2010 38
  49. 49. Don’t Expand “Lol” – Mixed-Case @Test public void shouldNotExpandLOLIfMixedCase() { String msg = "Do not expand Lol please"; when(session.getNextMessage()).thenReturn(msg); MessageExpander expander = new MessageExpander(session); assertThat(expander.getNextMessage(), equalTo(msg)); } This forced me to stop using Pattern.CASE_INSENSITIVE flag in pattern compilation. Only use “LOL” or “lol” for replacement criteria. public String getNextMessage() { String msg = session.getNextMessage(); return Pattern.compile("LOL|lol").matcher(msg) .replaceAll("laughing out loud"); } Monday, May 10, 2010 39
  50. 50. Don’t Expand “LOL” If Inside Word @Test public void shouldNotExpandLOLIfInsideWord() { String msg = "Do not expand PLOL or LOLP or PLOLP please"; when(session.getNextMessage()).thenReturn(msg); MessageExpander expander = new MessageExpander(session); assertThat(expander.getNextMessage(), equalTo(msg)); } The pattern matching is now modified to use spaces around each variation of valid LOL shorthand. return Pattern.compile("sLOLs|slols").matcher(msg) .replaceAll("laughing out loud"); Monday, May 10, 2010 40
  51. 51. Expand “LOL” If Not Inside Word @Test public void shouldExpandIfSurroundingCharactersAreNotLetters() { when(session.getNextMessage()).thenReturn("Expand .lol! please"); MessageExpander expander = new MessageExpander(session); assertThat(expander.getNextMessage(), equalTo("Expand .laughing out loud! please")); } The final implementation of pattern matching code looks as follows. return Pattern.compile("bLOLb|blolb").matcher(msg) .replaceAll("laughing out loud"); Monday, May 10, 2010 41
  52. 52. Merciless Refactoring Refactoring: a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.* Merciless: having or showing no [mercy - showing great kindness toward the distressed] Relieve your distressed code through kindness and disciplined restructuring * From http://www.refactoring.com/ Monday, May 10, 2010 42
  53. 53. Where to Start Refactoring? Does this change directly affect feature I am working on? Would change add clarity for feature implementation? Will change add automated tests where there are none? If “yes” to any question above, ask following question to decide if you should work on it now: At first glance, does refactoring look like a large endeavor involving significant portions of the software’s components? Monday, May 10, 2010 43
  54. 54. When to Stop Refactoring? Am I refactoring code not directly affected by feature? Is other code directly affected by feature I am working on that has not been refactoring sufficiently? If refactoring is exploding feature estimate given to Customer then I should bring it up to Team to decide how we should progress If Team decides that refactoring can be absorbed into current iteration without affecting delivery on our commitments then continue refactor Monday, May 10, 2010 44
  55. 55. Automate Testing to Support Refactoring cannot be done effectively without automated tests surrounding code Start by creating automated test which fails If difficult to create at unit level look at automated acceptance tests from functional perspective Over time look for ways to create automated unit tests Monday, May 10, 2010 45
  56. 56. Automated Tests Application Monday, May 10, 2010 46
  57. 57. Automated Tests Application New Feature Monday, May 10, 2010 46
  58. 58. Automated Tests Application New Design Changes Feature Monday, May 10, 2010 46
  59. 59. Automated Tests Application Automated New Design Changes Regression Test Run Feature Monday, May 10, 2010 46
  60. 60. Automated Tests Application Automated New Design Changes Regression Test Run Feature Monday, May 10, 2010 46
  61. 61. Principles of Agile Software Quality The system always runs No code is written without a failing test Zero post-iteration bugs * From “Flawless Iterations” presented by Alex Pukinskis at Agile 2005 Monday, May 10, 2010 47
  62. 62. Quality Debt A lack of quality, either technical or functional, will lessen value per feature added over time Monday, May 10, 2010 48
  63. 63. Accrual of Quality Debt with Releases Monday, May 10, 2010 49
  64. 64. Break/Fix Only Prolongs the Agony Monday, May 10, 2010 50
  65. 65. Effect of Project Constraints on Monday, May 10, 2010 51
  66. 66. Effect of Project Constraints on Monday, May 10, 2010 51
  67. 67. Acceptance Test-Driven Development Monday, May 10, 2010 52
  68. 68. A Fit Case Study Cost reduction using Fit for test automation and data conversion Monday, May 10, 2010 53
  69. 69. Manual Regression Testing Testing was taking 75 person hours during 2 full test runs consisting of: Comprehensive manual regression testing Data conversion and validation Cost for testing was $17,000 each iteration Monday, May 10, 2010 54
  70. 70. Introducing Fit into Testing Process After 8 iterations team had introduced healthy amount of Fit fixtures and automated tests Reduced 70+ hour test runtime down to 6 hours which now included: Fit automated regression testing Data conversion and validation automated with Fit fixtures Reduced cost of testing each iteration from $17,000 to $7,000 Monday, May 10, 2010 55
  71. 71. Configuration Management Debt Creating unpredictable and error-prone release management Monday, May 10, 2010 56
  72. 72. Traditional Source Control Monday, May 10, 2010 57
  73. 73. Traditional Source Control Main Branch Monday, May 10, 2010 57
  74. 74. Traditional Source Control Code Complete Version 1 Integrate for Branch Version 2 Main Branch Monday, May 10, 2010 57
  75. 75. Traditional Source Control Code Complete Version 1 Integrate for Branch Version 2 Debt Main Branch Death March Monday, May 10, 2010 57
  76. 76. Traditional Source Control Code Complete Version 1 Integrate for Branch Version 2 Debt Main Branch Death March { Debt accrues quickly within stabilization periods Monday, May 10, 2010 57
  77. 77. Flexible Source Control Management Monday, May 10, 2010 58
  78. 78. Flexible Source Control Management Main Branch Monday, May 10, 2010 58
  79. 79. Flexible Source Control Management Version 1 Main Branch Monday, May 10, 2010 58
  80. 80. Flexible Source Control Management Version 1 Version 2 Main Branch Monday, May 10, 2010 58
  81. 81. Flexible Source Control Management Version 1 Version 2 Main Branch { Not Easy! Must have proper infrastructure to do this. Monday, May 10, 2010 58
  82. 82. Continuous Integration Monday, May 10, 2010 59
  83. 83. Scaling to Multiple Integrations Monday, May 10, 2010 60
  84. 84. Design Debt Design decays when not attended to so design your software continually Monday, May 10, 2010 61
  85. 85. * Abuse User Stories Implement Security for User Information * From “User Stories Applied” presented by Mike Cohn Agile 2006 Monday, May 10, 2010 62
  86. 86. * Abuse User Stories Implement Security As a Malicious Hacker I for User Information want to steal credit card information so that I can make fraudulent charges * From “User Stories Applied” presented by Mike Cohn Agile 2006 Monday, May 10, 2010 62
  87. 87. Platform Experience Debt Silos of knowledge and increased specialization will increase cost of maintenance over time Monday, May 10, 2010 63
  88. 88. Team Configuration Patterns Virtual Architect Pattern Integration Team Pattern Component Shepherd Pattern Team Architect Pattern Monday, May 10, 2010 64
  89. 89. Virtual Architect Pattern Enterprise Planning Monday, May 10, 2010 65
  90. 90. Virtual Architect Pattern Pros Share architecture ideas and needs across teams Based on verbal communication Cons Usually singles out special Team Member role Could lead to top-down architecture decisions IT may gain extensive influence and begin to run Product Monday, May 10, 2010 66
  91. 91. Integration Team Pattern All features are Integrate implemented Features and integrated every iteration Feature Development Monday, May 10, 2010 67
  92. 92. Integration Team Pattern Pros Reduces complexity on Feature Teams Forces delivery from Integration Team instead of interface and deployment designs Cons Perpetuates specialized roles Don’t always work on highest value Product Backlog items Monday, May 10, 2010 68
  93. 93. Component Shepherd Pattern Monday, May 10, 2010 69
  94. 94. Component Shepherd Pattern Pros Share more knowledge within organization to minimize platform experience debt Work on highest value Product Backlog items Cons Not always optimal as using individual knowledge Difficult to learn multiple systems across Teams Monday, May 10, 2010 70
  95. 95. Team Architect Pattern Monday, May 10, 2010 71
  96. 96. Team Architect Pattern Pros Team owns architecture decisions Decisions are made close to implementation concerns Cons May not have appropriate experience on Team Team could get “stuck” on architecture decisions Monday, May 10, 2010 72
  97. 97. What is possible? High quality can be attained and should enable accelerated feature delivery at same time. Monday, May 10, 2010 73
  98. 98. A Story: Field Support Application 2000+ users access application each day Application supports multiple perspectives and workflows from Field Support Operations to Customer Service Team of 5 people delivering features on existing Cold Fusion platform implementation Migrating to Spring/Hibernate in slices while delivering valuable features 36 2-week Sprints, 33 production releases, and only 1 defect found in production So, what was the defect you say? Let me tell you… Monday, May 10, 2010 74
  99. 99. Lets wrap this up... What should I take away from this? Monday, May 10, 2010 75
  100. 100. Principles for Managing Software Debt Maintain one list of work Emphasize quality Evolve tools and infrastructure continually Improve system design always Share knowledge across the organization And most importantly, get the right people to work on your software! Monday, May 10, 2010 76

×