Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Refactoring, Emergent Design & Evolutionary Architecture

1,939 views

Published on

Describes Agile Design principles and practices in-depth, including refactoring, restructuring, clean-code, simple code, technical debt, incremental design, design principles, design patterns, emrgenet design, domain-driven design, evolutionary architecture.
by Brad Appleton

Published in: Software

Refactoring, Emergent Design & Evolutionary Architecture

  1. 1. Refactoring, Emergent Design & Evolutionary Architecture by Brad Appleton Created July 2009 (last updated, October 2010) ing Ref or Ref ing act or act ing ing act or act ing Ref or Ref ing or Ref ing act
  2. 2. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton 2 Presenter Information Brad Appleton 25+ years in industry – last 20 w/large Telecom. Agile Leader, Coach & Manager  Scrum, XP, SAFe, LeSS, DAD ALM/SCM/DevOps Solution Architect  Git, JIRA, Confluence, TFS, RTC, Jenkins, CI/CD Leading adoption & scaling of Agile development for large organization since 2000. Co-author: Software Configuration Management Patterns (Addison-Wesley 2002), Agile CM Environments (column in CM Journal), Reviewer for The Agile Journal
  3. 3. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Outline • Part I: Refactoring for Agility  Refactoring Overview  How to Refactor  Refactoring to Patterns & Principles • Part II: Refactoring @ Scale  Emergent Design  Technical Debt  Restructuring  Evolutionary Architecture  Incremental Design • Part III: Special Topics & Further Info  Refactoring Legacy Code  Refactoring Databases & Web Content  Refactoring Test Code  Additional Resources (Books & Articles) 3
  4. 4. Part I Refactoring for Agility ing Ref or Ref ing act or act ing ing act or act ing Ref or Ref ing or Ref ing act
  5. 5. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Outline for Part I • Overview of Refactoring  Refactoring Defined  Refactoring Myths & Misconceptions  The Practice of Refactoring  Why Refactor? • How to Refactor  Simple Code & Clean Code  The Refactoring Cycle  Code Smells & Refactorings  When to Refactor? • Refactoring to Patterns & Principles  Design Attributes / Code Qualities  Design Principles & Design Patterns  Other Acronyms of Simple/Agile Design • Summary & Resources for Part I 5
  6. 6. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Overview of Refactoring • Identifies design-maintenance issues (“code smells”) that typically represent violations of known principles of good design. • Incrementally and iteratively applies a set of design improvement techniques (“refactorings”). • The goal is to minimize complexity & duplication in order to maximize simplicity & ease-of-change. • Encourages the “right” design details to emerge “just-in-time” with minimal guesswork/rework. • Scaling-up includes the use of periodic restructuring, initial & incremental design (“just enough”), and evolutionary architecture. 6
  7. 7. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Refactoring Defined Refactoring (noun): a change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behavior. Refactoring (verb): to restructure software by applying a series of refactorings without changing its observable behavior. — Martin Fowler, Refactoring: Improving the Design of Existing Code “A change to the system that leaves its behavior unchanged, but enhances some nonfunctional quality – simplicity, flexibility, understandability, performance.” — Kent Beck, Extreme Programming Explained 7
  8. 8. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Refactoring Distilled Refactoring is a disciplined technique for the restructuring of an existing body of code, altering its internal structure without changing its external behavior.  Its heart is a series of small behavior preserving transformations.  Each transformation (called a ‘refactoring’ ) does only a little, but a sequence of transformations can produce a significant restructuring.  Since each refactoring is small, it's less likely to go wrong.  The system is kept fully working after each small refactoring, reducing the chances that a system can get seriously broken during the restructuring. Source: Martin Fowler, www.refactoring.com 8
  9. 9. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Refactoring Myths 9
  10. 10. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Refactoring is NOT … “Rework” – redesigning things that could, and should, have been designed correctly in the first place. Design “gold-plating” – work that adds no business value, and merely serves to stroke the egos of perfectionists who are out of touch with business reality. Miscellaneous code “tidying” – the kind that is “nice to have,” but should only happen when the team has some slack-time, and is a luxury we can do without, without any serious consequences. A license to “hack” – avoiding any and all initial design & analysis and instead jumping straight to coding with no “real” design. Reengineering – large-scale restructuring that requires a concerted effort over the course of several weeks/months to re-write or re- architect significant parts of the system. Derived from: Patrick Wilson, Continuous Refactoring and the Cost of Decay 10
  11. 11. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Refactoring IS … A systematic approach to source-code “hygiene” that minimizes the chances of introducing bugs Improving the design of the code after it has been written A behavior-preserving transformation of source-code structure The process of simplifying & consolidating a work-product by making several, small, successive revisions focused on:  preserving correctness,  removing redundancy,  revealing thoughts & intentions, and …  improving clarity & conciseness. A disciplined way of making changes while exposing the project to significantly less risk. An effective means to address the economic reality of software growth/complexity by reducing & amortizing its cost throughout the daily business of development & maintenance activities. 11
  12. 12. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton The Practice of Refactoring The process of refactoring involves:  the removal of duplication & dependencies  the simplification of complex logic/structure,  and the clarification of unclear code & design intentions Each step is small and simple, even simplistic.  You move a field from one class to another, pull some code out of a method to make into its own method, and push some code up or down a hierarchy. To refactor safely, you must run tests [preferably automated] to ensure your changes didn't break anything.  You will have more confidence to refactor and make other design changes if you can quickly run automated tests. Refactoring in small steps helps prevent introducing defects.  Most refactorings take only seconds or minutes to perform.  Even large restructurings are implemented in small steps. Sources: Martin Fowler, Joshua Kerievsky 12
  13. 13. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Why Refactor? Lots of reasons …  Make it easier to add new code  Improve the design of existing code  Increase readability/understanding of code  Make coding less annoying/frustrating  Prevent “design decay”  Clean-up/Simplify messy code  Find & fix design-principle violations  Reduce debugging time  Incorporate learning about the application 13
  14. 14. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton How to Refactor The overarching goal of refactoring is to: • minimize complexity & duplication in order to • maximize simplicity & ease-of-change. When we refactor we are striving to produce code that is clean and simple. Clean Code follows “Uncle” Bob Martin’s The Boy Scout Rule:  When touching any code, leave the codebase cleaner than you found it! Simple Code follows Kent Beck’s Four Rules of Simple Code … 14
  15. 15. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Simple Code 15
  16. 16. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton The Rules of Simple Code Simple Code … 1. Passes all the tests 2. Contains no duplication 3. Clearly reveals & expresses the developer’s intent 4. Minimizes the number and size of entities  classes/modules  methods/subroutines Note: these are in order of highest priority first! 16
  17. 17. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Clean Code 17
  18. 18. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton The Rules of Clean Code Keeping the Code “Clean” means ...  We must continuously simplify our code (following the rules of simple code)  We must continuously clean-up the code, and not tolerate messes, spills, or “smells” in our code.  We must not regress into bad habits. —Joshua Kerievsky, Introduction to Refactoring to Patterns • Rules for Clean Code come from the book of the same name by Robert C. Martin (a.k.a. “Uncle Bob”) et.al.  Clean Code: A Handbook of Agile Software Craftsmanship • Also see Uncle Bob’s Clean Code Tip of the Week series of articles online 18
  19. 19. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton The Refactoring Cycle 19
  20. 20. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton The Refactoring Cycle Check for “Code smells” Refactor Re-test After newly added/changed code passes its tests, you: 1. Check it for “Code smells” 2. When you find one, apply the specific refactoring that resolves it! 3. Re-run automated tests to ensure nothing broke (fix any “breakage” immediately) 4. Repeat until the affected code is smell-free! 20
  21. 21. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Code Smells 21
  22. 22. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Code Smells “Any symptom in the source code of a computer program that indicates something may be wrong. It generally indicates that the code should be refactored or the overall design should be reexamined.” • A Code Smell is a hint that something might be wrong, not a certainty. • Major approaches to programming “hygiene”:  Pragmatic: code smells should be considered on a case by case basis  Purist: all code smells should be avoided, no exceptions  Possible bad practice to a pragmatist, but a sure sign of bad practice to a purist • Beware of Code Stenches 22
  23. 23. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Code Smell Examples • Duplicate code or Duplicated method - a code, method, function, or procedure that is very similar to another • Large method or class - has grown too large • Lazy class - a class that does too little • Feature envy - a class that uses methods of another class excessively • Inappropriate intimacy - a class that has dependencies on implementation details of another class • Refused bequest - a class that overrides a method of a base class in such a way that the contract of the base class is not honored by derived class • Contrived Complexity - forced usage of overly complicated design patterns where simpler design would suffice • Code not actually ever used 23
  24. 24. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Categories of Refactorings Small Refactorings  (De)Composing methods  Moving features between objects  Organizing data  Simplifying conditional expressions  Dealing with generalization  Simplifying method calls Larger Refactorings/Restructurings  Tease apart inheritance  Extract hierarchy  Convert procedural design to objects  Separate domain from presentation Each category contains as many as a dozen or more refactorings, most of which are catalogued at http://refactoring.com/catalog/ 24
  25. 25. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Refactorings Some refactorings from real projects  Add a Parameter  Extract Method  Inline Method  Inline Class  Extract Interface  Move Field  Move Method  Parameterize Method  Rename Method  Method Parameter to Constructor Parameter Composite Refactorings (often to Design Patterns) See http://refactoring.com/catalog/ for an up-to-date list (and the “Refactoring to Patterns” catalog too) 25
  26. 26. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton What to do if …? I spot a “smell” that is not already known or catalogued?  Convince yourself it really is a “smell” by trying to identify its ill-effects and/or the design principle it appears to violate There is no specific known/catalogued “refactoring” for what I think I need?  Check again, just to be sure you didn’t miss it  Consult a co-worker  Use known design principles and/or software modifiability tactics to guide your judgment  Check for known design patterns to resolve that specific problem for your particular scenario 26
  27. 27. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton When to Refactor?  While adding functionality  While fixing a bug  While reviewing code  After coding the same/similar thing for the third time (to “factor out” the duplication) A.k.a., The Rule of Three: 3 strikes and you refactor.  After the third time you deferred refactoring a change, for any reason [The Rule of Three, again]  Before the end of the iteration if you haven’t been following The Rule of Three 27
  28. 28. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Refactoring Continually “It's best to refactor continuously (as a normal part of each coding activity) rather than in phases or increments.  When you come across code that is in need of improvement, improve it!  On the other hand, if you must finish a feature before a demo scheduled for tomorrow, finish the feature and refactor later. Business is well served by continuous refactoring, yet refactoring must coexist harmoniously with business priorities.” ―Joshua Kerievsky, Introduction to Refactoring to Patterns “In my view, refactoring is not an activity you set aside time to do separately from implementation activities. Refactoring is something you do all the time in little bursts.” ―Martin Fowler, Refactoring: Improving the Design of Existing Code 28
  29. 29. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton When NOT to Refactor?  When the build is broken or tests don’t pass  When it would compromise meeting an impending deadline or commitment  When the code in question really just needs to be re-written “from scratch”  When it would modify code/interfaces that could significantly impact/break other work:  Published/public interfaces and protocols  Database schemas/tables/operations Sometimes we must defer refactoring for later and/or plan for subsequent restructuring 29
  30. 30. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Refactoring to Patterns & Principles Software Design Principles & Design Patterns are the underlying foundation for Refactoring:  Code smells (a.k.a “code pathologies”)  Signal a possible violation of design principles  Suggest which refactoring may be needed  Refactorings  Correct a design principle violation (at least partially)  Converge toward common design patterns  Design Patterns  Reconcile forces among conflicting design concerns  Restore balance between competing design principles  Design Principles  Lead us to attain desired design qualities/attributes 30
  31. 31. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Design Attributes/Code Qualities 31
  32. 32. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Design Attributes/Code Qualities Qualities of Highly Maintainable Software:  Loose Coupling & High Cohesion  Hierarchy (Structural Decomposition)  Abstraction, Encapsulation & Modularity  Sufficiency, Parsimony and Primitiveness  Readability  Testability  Modifiability  Serviceability 32
  33. 33. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Design Principles 33
  34. 34. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Design Principles: SOLID, SoC, DRY, Shy The SOLID Principles of Object-Oriented Design: SRP : The Single Responsibility Principle OCP : The Open-Closed Principle LSP : The Liskov Substitution Principle ISP : The Interface Segregation Principle DIP : The Dependency Inversion Principle The SoC Principle: Separation of Concerns — separate interface from implementation, policy from mechanism, behavior from construction, commands from queries, levels of abstraction, ... The DRY Principle: Don’t Repeat Yourself (Eliminate Duplication), Single Point of Truth (SPOT) The “Structure-Shy” Principle: (“Tell, Don’t Ask!”), The Law of Demeter, Principle of Least Assumed Knowledge 34
  35. 35. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Other Acronyms of Simple/Agile Design OAOO – Say Things Once And Only Once (restatement of the DRY principle) DTSTTCPW – Do The Simplest Thing That Could Possibly Work! (restatement of the KISS principle) YAGNI – You Aren’t Gonna Need It! The LRM Principle: Defer Commitment of Irreversible Decisions to the Last Responsible Moment! BDUF – Big Design Up-Front! (vs. JEDI) JEDI – Just Enough Design Initially/In-front! DDD – Domain-Driven Design 35
  36. 36. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Design Patterns 36
  37. 37. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Design Patterns A Design Pattern is … • A recurring solution to a common problem in a given context and system of forces. • A named "nugget" of instructive insight, conveying the essence of a proven solution to a recurring problem in a given context amidst competing concerns. • A successfully recurring "best practice" that has proven itself in the "trenches." • A literary format for capturing the wisdom and experience of expert designers, and communicating it to novices. Design patterns … • Establish a common terminology and professional vocabulary for discussing recurring design problems & their reusable solutions • Raise the “level of thinking” in our communication and understanding about software design • Are context-specific applications of general design principles 37
  38. 38. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Design Patterns Examples Examples of well-known Design Patterns: • Creational: Abstract Factory, Builder, Factory Method, Prototype, Singleton • Structural: Adapter, Bridge, Composite, Decorator, Façade, Flyweight, Proxy • Behavioral: Chain of Responsibility, Command, Interpreter, Iterator, Mediator, Memento, Observer, State, Strategy, Template Method, Visitor Patterns & pattern catalogs are available for: • Concurrent, Parallel & Distributed Systems, Network Architecture, User-Interaction Design, Secure Programming, Database Access, Real-time/Embedded, Information Design, Software Product-Lines, J2EE (and core J2EE), dotNET, EA, SOA, and Many others 38
  39. 39. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Summary: Refactoring for Agility  Successively applies small behavior-preserving transformations to eliminate code smells  Based on proven design principles and patterns for achieving maintainability & modifiability  Good automated testing is a prerequisite  Refactoring is not rewriting, rework or restructuring  With refactoring, we continuously invest nominal effort to reduce the risk & cycle-time of changes  The goal is to minimize complexity & duplication in order to maximize simplicity & ease-of-change. 39
  40. 40. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Summary: Refactoring for Agility When practiced in a highly disciplined manner, refactoring promotes:  Sufficient functionality  Simple & clean code  Supple design  Serviceable software  Sustainable team velocity 40
  41. 41. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Resources: Code Smells • Introduction to Code Smells and Refactoring from TestingEducation.org • Code Smells and Refactoring to Patterns, by Josh Kerievsky • Refactoring and Code Smells • The original WikiWiki CodeSmells page • Martin Fowler's Code Smells page • A Catalogue of Bad Smells in Code, by Martin Fowler and Kent Beck • A Taxonomy of Bad Code Smells • CodingHorror.com's list of Code Smells • Smells to Refactorings Quick Reference, by Josh Kerievsky • Gene Garcia's mapping of CodeSmells to Refactorings • Using Static Analysis Tools to Identify Code Smells, IBM developerWorks article by Paul Duvall 41
  42. 42. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Resources: Design Principles • S.O.L.I.D. Principles e-Book (free) http://www.lostechies.com/content/pablo_ebook.aspx • An O-O Primer, http://www.rgoarchitects.com/Files/ooprimer.pdf • The Principles of Object-Oriented Design: http://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod • Design Principles and Patterns, http://www.objectmentor.com/resources/articles/Principles_and_Patterns.pdf and http://www.oodesign.com/ • The Art of SoC (Separation of Concerns), http://www.ctrl-shift- b.com/2008/01/art-of-separation-of-concerns.html • Composed Method & Single-Level of Abstraction, http://www.ibm.com/developerworks/java/library/j-eaed4.html • Tell, Don’t Ask, http://www.pragprog.com/articles/tell-dont-ask • O-O in One Sentence: Keep it Shy, DRY, and Tell the Other Guy! http://media.pragprog.com/articles/may_04_oo1.pdf 42
  43. 43. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Resources: Design Patterns Online Resources: • Wikipedia on Design Patterns • The Patterns Home page: hillside.net/patterns • Design Patterns Reference and Patterns Tutorials • Patterns & Software: Essential Concepts & Terminology Books: • Design Patterns, by the “Gang of Four” (the book that started it all) • POSA Series of books on Pattern-Oriented Software Architecture • Head First Design Patterns, from the O’Reilly Head First Series • Refactoring to Patterns; by Joshua Kerievsky • Agile Software Development, Principles, Patterns, and Practices by Robert C. Martin • Implementation Patterns; by Kent Beck • Design Patterns Explained (2ed); by Alan Shalloway, James Trott 43
  44. 44. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Resources: Other Agile Design Slogans • XP Simplicity Rules, http://c2.com/cgi/wiki?XpSimplicityRules • Essential XP: Simple Design, http://www.xprogramming.com/xpmag/expEmergentDesign.htm • DTSTTCPW - http://c2.com/cgi/wiki?DoTheSimplestThingThatCouldPossiblyWork • OAOO - http://c2.com/cgi/wiki?OnceAndOnlyOnce • YAGNI - http://en.wikipedia.org/wiki/You_Ain%27t_Gonna_Need_It • LRM - http://stevesmithblog.com/blog/delaying-decisions/, http://vig.pearsoned.co.uk/samplechapter/0321150783.pdf • BDUF - http://en.wikipedia.org/wiki/Big_Design_Up_Front • JEDI - http://www.featuredrivendevelopment.com/node/507 • DDD - http://www.domaindrivendesign.org/ 44
  45. 45. Part II Refactoring @ Scale ing Ref or Ref ing act or act ing ing act or act ing Ref or Ref ing or Ref ing act
  46. 46. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Outline for Part II • Scaling-up! • Emergent Design • Technical Debt  Technical Debt – Tips & Truths  Restructuring Technical Debt • Restructuring  Refactoring vs. Restructuring  Restructure Periodically  Architecture Smells & Modifiability Tactics  When to Plan for Restructuring? • Evolutionary Architecture  Incremental Design  JEDI (Just Enough Design Initially)  Supple Design & Domain-Driven Design • Resources for Part II 46
  47. 47. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Scaling-Up
  48. 48. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Scaling-Up To scale refactoring for larger projects, additional techniques & concepts must be added to the mix.  Note that this is “in addition to” (not “instead of”) Refactoring In-the-Small Refactoring @ Scale (adds the following) Small, Fast & Frequent Refactorings Larger, Periodic & Planned Restructurings Emergent Design Incremental Design & Evolutionary Architecture Deferred Refactoring Restructuring & Technical Debt Code Smells Architecture Smells Design Principles & Patterns Software Modifiability Tactics Simple/Clean Code Supple/Domain-Driven Design
  49. 49. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Emergent Design 49
  50. 50. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Emergent Design Emergent Design is a fancy name for the resulting design that “emerges” from the synergy of combining Refactoring together with TDD, Continuous Integration and Automated Testing. “As a result, you’ll find the balance of work changes.  Design, rather than occurring all up front, occurs continuously during development.  The cumulative effect of these small changes can radically improve the design.  You learn from building the system how to improve the design.  The resulting interaction leads to software with a design that stays good as development continues.  It is the exact reverse of the normal notion of software decay.” ― Martin Fowler, Refactoring: Improving the Design of Existing Code A lack/lapse of discipline in any of these technical practices will result in the accrual of technical debt! 50
  51. 51. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Technical Debt 51
  52. 52. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Technical Debt [a.k.a. Design Debt] Technical debt occurs when our software becomes difficult or risky to change, and takes increasingly more time and effort to evolve. • It represents the accumulated amount/cost of rework that will be necessary to correct and/or recover from the deviation between:  the current design of the system versus …  a design that is minimally complex yet sufficiently complete to ensure correctness & consistency for timely delivery. • This effort grows more than linearly over time as a system becomes bigger and more complex. • The economic impact of technical debt is directly related to the cost of complexity and its resulting “friction” against the velocity of the development team. 52
  53. 53. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Technical Debt – Real Costs The New Push to Measure Software’s True Cost, from ComputerWorld.com, October 2010: • The idea that software acquires a "technical debt" that is paid in real dollars is getting new attention and research:  The National Science Foundation approved a $465,000 research grant last year on technical debt  Gartner Research just released its study on the subject of "IT debt“, and puts the current IT debt bill at $500 billion worldwide, and says that it will double in five years to $1 trillion.  A company that makes software quality tools, Cast, just released a study gleaned from customer software evaluations that puts the cost of technical debt at $2.82 per line of code. For the average-size software application of 374,000 lines of code, this amounts to just over $1 million in technical debt. 53
  54. 54. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Technical Debt – Tips & Truths • Incurring technical debt means your velocity slows down and you will deliver less value • The cost of getting out-of-debt is compounded over time: the longer you wait, the faster it grows • If you plan to incur technical debt, the persons responsible must have a workable plan to pay it off! • “Interest only” payments won’t improve things • Pay early, pay often, and pay-as-you-go. (The only other options are bankruptcy or death.) • Remember: Those with the worst debt problems often have the most difficulty imagining a life without borrowing! derived from http://agileinaflash.blogspot.com/2009/02/technical-debt.html 54
  55. 55. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Restructuring Technical Debt If we accrue a non-trivial amount of technical debt, we can’t simply “refactor” it away.  Paying it off typically requires restructuring efforts (or even reengineering) that must be planned.  Iteration plans must accommodate specific tasks for these restructuring efforts (or even be dedicated to restructuring).  Ignoring it, or deferring it for very long is not a viable option! “All successful software gets changed. So if we think we’re working on code that will be successful … we need to keep it easy to change. Anything that makes code difficult to change is technical debt. Just like any other debt, the cost of paying off technical debt gets more and more expensive over time. … Technical debt drives the total cost of software ownership relentlessly higher … eventually we will have to pay it off or the system will go bankrupt.” ―Mary Poppendieck, Leading Lean Software Development 55
  56. 56. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Restructuring 56
  57. 57. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Overview of Restructuring • Identifies higher-level issues (“architecture smells”) that typically represent violations of known principles of good software architecture & design. • Periodically applies larger-scale “refactorings” and/or many small refactorings that were previously deferred. • The goal is to “pay down technical debt” in order to limit the increasing costs of accumulated complexity. • Typically requires a concerted effort that must be separately planned. • Uses not only design patterns/principles, but also architectural patterns/principles, as well as software modifiability tactics. 57
  58. 58. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Refactoring vs. Restructuring The following are not valid examples of refactoring: ● The system being broken (unbuildable) for a couple of days while someone is “refactoring” ● Reworking the structure of an entire document or web site/portal ● A concerted effort by one or more people over several days/weeks to rework and consolidate the code/design structure All of the above are restructuring: Restructuring is any rearrangement of parts of a whole. It's a very general term that doesn't imply any particular way of doing the restructuring. Refactoring is a very specific technique, founded on using small behavior-preserving transformations (themselves called refactorings). ―Martin Fowler, Refactoring Malapropism 58
  59. 59. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Restructure Periodically • Restructuring is often associated with absent or neglectful refactoring and/or design. • But … Any large software project spanning multiple teams eventually needs restructuring.  Even in the presence expert-level architecture, design & continuous refactoring  This is just a reality of software evolution/entropy • Therefore … Large software projects should assume that periodic restructuring will be necessary, and should plan accordingly to:  Clean-up accumulated code-smells and apply numerous refactorings that were deferred but are now sorely needed,  Address architecture smells by applying restructurings, patterns and modifiability tactics that have broader impact. 59
  60. 60. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Architecture Smells 60
  61. 61. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Architecture Smells Smells in Dependency Graphs • Obsolete Classes • Tree-like, Static Cycles • Visibility of Dependency Graphs Smells in Inheritance Hierarchies • Type Queries, List-like Hierarchy • No Subclass Method Redefinition • No Polymorphic Assignments • Parallel or Too Deep Hierarchy Smells in Packages • Unused Packages • Cyclic Package Dependencies • Packages Too Small / Large • Package Hierarchies Unbalanced • Package Hierarchies Not Clearly Named Smells in Subsystems • No Subsystems • Cycles between Subsystems • Subsystem Too Large / Small • Subsystem-API Bypassed • Subsystem-API Too Large • Overgeneralization Smells in Layers • No Layers, Too Many Layers • Strict Layers Violated • Upward References Between Layers (Cycles between Layers) • Inheritance between Protocol- Oriented Layers • References between Vertically Separated (non-adjacent) Layers Source: Refactoring in Large Software Projects, by Martin Lippert & Stefan Roock 61
  62. 62. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Software Modifiability Tactics 62
  63. 63. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Software Modifiability Tactics Localize Changes (increase cohesion)  Maintain Semantic Coherence  Anticipate Expected [types of] Changes  Generalize the Module  Limit Possible Options  Abstract Common Services Prevent Ripple Effects (reduce coupling)  Hide Information  Maintain Existing Interfaces  Restrict Communication Paths  Use an Intermediary Defer Binding-time (defer decision-making)  Run-time Registration  Configuration Files  Polymorphism/Delegation  Component Replacement  Adhere to Defined Protocols Source: SEI Technical Report CMU/SEI-2007-TR-002 63
  64. 64. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton When to Consider/Plan Restructuring? Consider whether restructuring effort is needed … • During Planning (of course)  Release planning / Sprint planning  Of “special” iterations (e.g., “spikes”, architecture, etc.)  Of stabilizing (“hardening”) iterations • During Retrospectives  Iteration retrospectives  Inter-team (“joint”) retrospectives [Scrum-of-Scrums level] • After Reviews  After a Sprint review  After a Technical/Release/Program review • After three iterations without any restructuring 64
  65. 65. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Evolutionary Architecture 65
  66. 66. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Evolutionary Architecture Software Architecture concerns infrastructure elements that must exist before you can begin execution.  Architecture is about things that are hard to change later, it is difficult to allow an architecture to emerge.  For large projects, this includes high-level organization of the system into functionality/elements that will be allocated to separate teams. However, just because we can't allow architecture to emerge doesn't mean that it can't evolve.  If we create an initial, flexible architecture and take special care to not create an irreversible decision, then we can allow it to evolve over time as new concerns appear. — Neal Ford, Evolutionary Architecture and Emergent Design Key techniques of Evolutionary Architecture include:  Deferring Irreversible Decisions to the “Last Responsible Moment” (LRM Principle)  Architectural “Spike” (a.k.a. Architectural “Deep Dive”)  Architecture Iteration and/or Spike Iteration 66
  67. 67. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Incremental Design Design evolves incrementally, iteration by iteration, based on current business priorities and discovered technical limitations. “Invest in the design only what is needed to comfortably support the current iteration.” —Kent Beck & Cynthia Andres Incremental Design … • Does not prohibit thinking about higher-level design. • Does encourage planning in detail only what will be constructed soon. • Focuses on “Just Enough, Just-In-Time” :  Specifying too much detail too soon causes more rework later.  But doing less now and saving the rest for later should not require significantly more work later than it would today.  We must do Just Enough Design Initially to attain the right balance of anticipation and adaptation. 67
  68. 68. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Just Enough Design Initially (JEDI) 68
  69. 69. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Just Enough Design Initially (JEDI) Initial design (before coding) is still necessary.  This use of “JEDI” was coined by Stephen Palmer as part of Feature-Driven Development (FDD) Basic rule of thumb to tell when “JEDI” is achieved:  At iteration-scope, when, after one pass through the iteration backlog, modeling in small groups does not produce any new classes or associations of real significance.  At task/TDD scope, when we have defined enough structure & interface(s) to know specifically what code to write/test, precisely where to write it, and exactly how to invoke it. Techniques of “the JEDI way” include:  Collaborative Domain Modeling and Color Modeling [from FDD]  Supple Design techniques & patterns  Domain-Driven Design (a.k.a. DDD -- see domaindrivendesign.org)  Design Blitz & other Agile Modeling techniques (see agilemodeling.com) 69
  70. 70. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Supple Design 70
  71. 71. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Supple Design & DDD Domain-Driven Design (DDD) approaches modeling the core logic of the software by focusing on the domain.  The basic idea is the design should directly reflect the core business domain and domain-logic of the problem to solve  This helps understanding the problem as well as the implementation and increases maintainability of the software. DDD uses common principles and patterns as "building blocks" to model & create a “supple design”  Supple: pliant, malleable, limber, yielding or changing readily.  The design is firm yet flexible, with structure and intent both clearly conveyed and deeply realized by the code. Patterns of Supple Design include:  Intention-Revealing Interfaces, Ubiquitous Language, Side-Effect- Free Functions, Assertions, Conceptual Contours, Standalone Classes, Closure of Operations, Declarative Style 71
  72. 72. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Resources: Evolutionary Architecture • Neal Ford's series of IBM developerWorks articles on Evolutionary Architecture and Emergent Design • Chris Sterling's book on Architecture in an Agile Organization • Presentations by Ryan Shriver at theagileengineer.com • Dean Leffingwell's writings on Agile Architecture • OOPSLA '09 workshop on "Architecture in an Agile World" • Software Architecture and Agile Software Development - An Oxymoron? by Philippe Kruchten • Agile Architecture - How much is enough?, by Eoin Woods • The Agile Architect site (including the role of the agile architect) • Agile Architecting by Erik Philippus • Scott Ambler's Scaling Agile Development via Architecture • Lean Software Architecture, Jim Coplien & Gertrud Bjornvig • Architecture Meets Agility, by Hakan Erdogmus • Systems Engineering and Architecting Challenges: Application to Agile Development, by Murray Cantor 72
  73. 73. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Resources: Emergent Design and Incremental Design • Emergent Design: The Evolutionary Nature of Professional Software Development, by Scott L. Bain • Neal Ford, Evolutionary Architecture and Emergent Design series of articles on IBM developerWorks • Is Design Dead? by Martin Fowler • The Mikado Method, by Daniel Brolund & Ola Ellnestam 73
  74. 74. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Resources: Restructuring • InfoQ: Michael Stal on Architecture Refactoring • Refactoring in Large Software Projects: Performing Complex Restructurings Successfully, by Martin Lippert, Stephen Roock (also an earlier version online) • The Mikado Method and Refactoring Large Software Systems • Object-oriented Reengineering Patterns, by S. Demeyer, S. Ducasse & O. Nierstrasz (freely downloadable online) 74
  75. 75. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Resources: Technical Debt • Technical Debt and Design Death, by Kane Mar and Michael James • Design debt economics, by John Elm, IBM developerWorks, June 2009 • The Agile Executive articles on Technical Debt, by Israel Gat • Managing Technical Debt and 10X Software Development - Technical Debt, by Steve McConnell • Managing Technical Debt; by Tom Brazier • Repaying Technical Debt; by Simon Baker • Software Debt and Technical Debt, by Chris Sterling, author of Managing Software Debt (also see video presentation) • Software Entropy: Don’t Tolerate Broken Windows, and Zero-Tolerance Construction by Andy Hunt and Dave Thomas • Monetizing Technical Debt, and other InfoQ articles on technical debt • Technical Debt - How not to ignore it! by Henrik Kniberg • Continuous Refactoring and the Cost of Decay, by Patrick W. Welsh 75
  76. 76. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Resources: Modifiability Tactics • Modifiability Tactics, SEI @ CMU Technical Report CMU/SEI- 2007-TR-002  http://www.sei.cmu.edu/publications/documents/07.reports/07tr002.html • Modifiability Tactics, Chapter 5 section 3 of Software Architecture in Practice  http://www.tar.hu/softarchpract/ch05lev1sec3.html • Understanding Architectural Patterns in Terms of Tactics and Models  http://www.sei.cmu.edu/news-at-sei/columns/the_architect/2007/08/architect-2007-08.htm • Design and Tactics, Lecture notes from an Aalborg University course on Database and Software Architecture  http://www.cs.aau.dk/%7Emly/dbsa07/L8_Design_and_Tactics.pdf 76
  77. 77. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Resources: Supple Design & DDD • Wikipedia page on DDD: http://en.wikipedia.org/wiki/Domain-driven_design • DDD Intro Article: http://www.infoq.com/articles/ddd-in-practice and http://www.typo3- media.com/blog/domain-driven-design-introduction.html • DDD – a Brief Introduction: http://www.typo3-media.com/blog/domain-driven-design-introduction.html • DDD: Supple Design Patterns http://www.cs.colorado.edu/~kena/classes/6448/s05/lectures/lecture30.pdf • DDD Pattern Summaries: http://domaindrivendesign.org/resources/what_is_ddd • Free mini-eBook: Domain-Driven Design Quickly http://www.infoq.com/minibooks/domain-driven-design-quickly • Yet another free mini-eBook: Step-by-Step Guide to DDD http://dddstepbystep.com/ • See also http://domaindrivendesign.org/ and http://www.infoq.com/domain-driven-design 77
  78. 78. Part III Special Topics and Further Information/Resources ing Ref or Ref ing act or act ing ing act or act ing Ref or Ref ing or Ref ing act
  79. 79. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Refactoring & Legacy Code From Michael Feathers’ book Working Effectively with Legacy Code (WELC) :  Legacy Code is defined as any existing code that does not have adequate automated test coverage. Recommends an incremental approach for legacy code:  Refactor & automate tests for legacy methods/subroutines whenever you are already “in the neighborhood” to modify them for an existing task in the current iteration.  Don’t attempt a comprehensive & thorough “legacy clean-up” all at once (no “magnum opus” grand-redesign-in-the-sky)  Break-up larger/significant “legacy clean-up” efforts into smaller chunks and defer to the next planned restructuring. 79
  80. 80. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Refactoring Databases See the following: • Refactoring Databases: Evolutionary Database Design, by Scott W. Ambler & Pramodkumar J. Sadalage • Refactoring SQL Applications, by Stephane Faroult, Pascal L'Hermite • Catalog of database Refactorings, by Scott Ambler et.al. • Evolutionary Database Design, by Martin Fowler • Agile Database Refactoring with Hibernate, by Gilad Buzi, Kelley Glenn, and Jonathan Novich. 80
  81. 81. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Refactoring Web Content and Test Code See the following: • Refactoring HTML: Improving the Design of Existing Web Applications, by Elliotte Rusty Harold • xUnit Test Patterns: Refactoring Test Code, by Gerard Meszaros 81
  82. 82. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Sample Test Code Refactorings Minimize Data:  Remove things from the fixture until we have a Minimal Fixture. Inline Resource:  Move contents of an external resource into fixture setup logic of the test. Make Resource Unique:  Make the name of any resources used by a test unique. Replace Dependency with Test Double:  Break dependency by replacing a depended-on component with a Test Double. Extract Testable Component:  Extract logic to test into a separate component that is designed for testability and is independent of the context in which it is run. Setup External Resource:  Create an external resource within the fixture setup logic of the test rather than using a predefined resource. Source: www.xunitpatterns.com 82
  83. 83. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Online Resources • The Refactoring homepage - www.refactoring.com • Refactoring to Patterns Catalog • Refactoring: Small Steps Guaranteed to Help You Clean Up Your Code, by C. Keith Ray. • Introductory Refactoring Articles • Podcasts, “Refactoring: Part 1 and Part 2,” a talk with Martin Lippert. • Refactoring Checklist, by Michael Nielsen. • Michael Stal on Architecture Refactoring, by Niclas Nilsson. • Introduction to Refactoring to Patterns, by Joshua Kerievsky. • Code Refactoring Guidelines, by Federico Cargnelutti. • Smells to Refactorings table, from wiki.java.net • Refactoring with Code Metrics, Andrew Glover • Refactoring Thumbnails, from www.refactoring.be • Continual Refactoring, by Paul Duvall • Refactoring blogs, books, forums, sample chapters, videos, tools and tutorials. 83
  84. 84. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Books Refactoring: Improving The Design of Existing Code; by Martin Fowler Refactoring to Patterns; by Joshua Kerievsky Emergent Design: The Evolutionary Nature of Professional Software Development, by Scott L. Bain Clean Code: A Handbook of Agile Software Craftsmanship; by Robert C. Martin Working Effectively with Legacy Code; by Michael Feathers xUnit Test Patterns: Refactoring Test Code, by Gerard Meszaros Domain-Driven Design: Tackling Complexity in the Heart of Software; by Eric Evans Implementation Patterns; by Kent Beck 84
  85. 85. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Books 85
  86. 86. Backup Slides [for reference material]
  87. 87. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton What Motivates us to Refactor? • Make it easier to add new code.  When adding new functionality to a system, we have a choice: a)implement without regard to how well it fits the existing design, b)modify existing design to easily accommodate the new feature.  With a), we incur design debt, which can be paid down later by refactoring [or more significant restructuring].  With b), we analyze what must change to best accommodate the new feature and then make the necessary changes. • Improve the design of existing code.  By continuously improving the design of code, we make it easier and easier to work with.  Continuous refactoring involves constantly sniffing for code smells and removing them immediately (or soon after)  If you get into the hygienic habit of refactoring continuously, you'll find that it is easier to extend and maintain code. Source: J. Kerievsky, Introduction to Refactoring to Patterns 87
  88. 88. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton What Motivates us to Refactor? [2] • Gain a better understanding of code.  Sometimes we look at code and have no idea what it does or how it works.  Even if someone could stand next to us and explain the code, the next person to look at it could also be totally confused.  If the code isn't clear, it's an odor that needs to be removed by refactoring, not by “deodorizing” the code with a comment. • Make coding less annoying/frustrating.  Sure, we can say we refactor to remove duplication, to simplify or clarify the code. But what actually compels us to refactor?  Sometimes it is emotions. Face it – it’s just plain frustrating to have to fix or extend code that is poorly structured, overly complex, and difficult to understand/navigate. Source: J. Kerievsky, Introduction to Refactoring to Patterns 88
  89. 89. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Refactoring and Evolution Working incrementally changes “I wish I had thought of this sooner” to “Now I get it; let's do it this way.” It is a fundamentally stronger position to be in. • Evolution requires change ― refactoring is a disciplined way of making change while exposing the project to significantly less risk. • It allows us to introduce design at a later period, especially through the concept of refactoring to open-closed. • Knowing you can do this reduces the ratio of design to over- design. — Scott L. Bain Emergent Design: The Evolutionary Nature of Professional Software Development, May 2008 89
  90. 90. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Test-Driven Refactoring Testing also plays an altogether different role in refactoring;  it can be used to rewrite and replace old code. A test-driven refactoring involves applying test-driven development to produce replacement code and then swap out old code for new code (while retaining and rerunning the old code's tests).  Just remember that the "reimplement and replace" technique, as performed by using test-driven refactoring, is another useful way to refactor.  While it tends to be most helpful when you're designing a new algorithm or mechanism, it may also provide an easier path than applying low-level or composite refactorings. Source: J. Kerievsky, Introduction to Refactoring to Patterns 90
  91. 91. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton 7 Deadly Symptoms of Rotting Design 1. Rigidity – make it hard to change 2. Fragility – make it easy to break 3. Immobility – make it hard to reuse 4. Viscosity – make it hard to do the right thing 5. Needless Complexity – over design 6. Needless Repetition – error prone 7. Not doing any Source: Robert C. Martin, http://www.rgoarchitects.com/Files/ooprimer.pdf & http://www.objectmentor.com/resources/articles/Principles_and_Patterns.pdf 91
  92. 92. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton The Laws of Software Evolution A series of 8 laws formulated by Lehman & Belady starting in 1974, including:  The Law of Continuing Change  The Law of Increasing Complexity  The Law of Continuing Growth  The Law of Declining Quality The laws can be summarized as follows:  A system that is being used undergoes continuing change or degrades in effectiveness.  A computer program that is changed, becomes less and less structured. The changes increase the entropy and complexity of the program. Source: Wikipedia on “Lehman’s Laws of Software Evolution” 92
  93. 93. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Dimensions of software complexity Higher technical complexity - Embedded, real-time, distributed, fault-tolerant - Custom, unprecedented, architecture reengineering - High performance Lower technical complexity - Mostly 4GL, or component-based - Application reengineering - Interactive performance Higher management complexity - Large scale - Contractual - Many stake holders - “Projects” Lower management complexity - Small scale - Informal - Single stakeholder - “Products” Defense MIS System Defense Weapon SystemTelecom Switch CASE Tool National Air Traffic Control System Enterprise IS (Family of IS Applications) Commercial Compiler Business Spreadsheet IS Application Distributed Objects (Order Entry) Small Scientific Simulation Large-Scale Organization/Entity Simulation An average software project - 5-10 people - 3-9 month duration - 3-5 external interfaces - Some unknowns & risks Embedded Automotive Software IS Application GUI/RDB (Order Entry) Source: Walker Royce 93
  94. 94. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Managing Software Complexity In No Silver Bullet, Fred Brooks defines two categories of complexity:  Essential complexity: This is the inherent complexity of the problem domain. There is no avoiding it. (It comes with the territory.)  Accidental complexity: This is the complexity of the solution. It is a byproduct of the systems, languages, and frameworks we use. (“We have met the enemy, and it is us!”) • In principle, accidental complexity can be reduced by changing the system.  Refactoring reduces accidental complexity! 94
  95. 95. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Accidental Complexity Three things tend to spawn accidental complexity. • Design debt -- just-in-time hacks to code because of schedule or other external pressures. • Duplication, violations of the DRY (Don't Repeat Yourself) principle.  Duplication is the single most insidious diminishing force in software development because it manages to creep into so many places without developers even realizing it..  Duplication harms projects because it resists attempts to make structural changes or refactor toward better code. • Irreversibility. Any decision you make that cannot be reversed will eventually lead to some level of accidental complexity.  Irreversibility affects both architecture and design, although its effects are both more common and more damaging at the architectural level. Try to avoid decisions impossible or cumbersome to reverse. Source: Neal Ford, Evolutionary Architecture and Emergent Design 95
  96. 96. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Why do we call it “Debt”? Ward Cunningham's financial metaphor of design debt works far more effectively than technical language (e.g., “refactoring”) to influence management. Due to ignorance or the urge to "not fix what ain't broke" too many developers & teams spend too little time paying down design debt.  if you don't pay off a debt, you incur late fees.  If you don't pay your late fees, you incur higher late fees.  The more you don't pay, the worse your fees and payments become.  Compound interest kicks in, and as time goes on, getting out of debt becomes an impossible dream. The same is true with design debt. Source: Joshua Kerievsky, Introduction to Refactoring to Patterns 96
  97. 97. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton What Causes Technical Debt? It is a difficult, delicate and dynamic balancing act to achieve the necessary and sufficient amount of design to implement only the essential complexity required by system. We lack perfect knowledge/certainty, and can’t always get it right the first time we try:  Sometimes we knowingly (and under pressure) do something the "quick & dirty" way, with the intent to “clean it up” later.  Sometimes we attempt too much to soon, delving into details of the requirements when we and our customers/users haven't yet learned enough about the true needs of the system.  Sometimes we unknowingly violate design principles, resulting in undesirable dependencies that make code or other work-products hard to change.  Sometimes we neglect to properly "tend" to the design and don't give it the necessary amount of ongoing care and feeding needed to keep it "fit" and "supple." 97
  98. 98. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Principal & Interest • Every developer becomes aware of the concept of technical debt, whereby you make compromises in your design for the sake of some external force, such as schedule pressure.  Technical debt resembles credit card debt: you don't have enough funds at the moment, so you borrow against the future.  Similarly, your project doesn't have enough time to do something right, so you hack a just-in-time solution and hope to use some future time to come back and retrofit it. • Unfortunately, many managers don't seem to understand technical debt, causing resistance to revisiting past work. Source: Neal Ford, Evolutionary Architecture and Emergent Design 98
  99. 99. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Software Entropy & Complexity Building software isn't like digging a ditch.  If you make compromises when you dig a ditch, you just get uneven width or unequal depth. Today's flawed ditch doesn't prevent you from digging a good ditch tomorrow.  But the software you build today is the foundation for what you build tomorrow. Compromises made now for the sake of expediency cause entropy to build up in your software. Entropy is a measure of complexity, and if you add complexity now because of a just-in-time solution to a problem, you must pay some price for that for the remaining life of the project. Source: Neal Ford, Evolutionary Architecture and Emergent Design 99
  100. 100. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Technical Debt & Interest • Let's say that you want to add new features to an existing, long-running project.  These new features have a certain inherent complexity to them.  However, if you have technical debt already, you must work around those compromised parts of the system to add new features. • Thus, the cost for additions mirrors the financial metaphor. • Figure 2 shows the difference between the effort required to add a new feature in a cleanly designed system (for example, one with little or no technical debt), versus a typical system that contains a lot of technical debt. Source: Neal Ford, Evolutionary Architecture and Emergent Design 100
  101. 101. Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton Technical Debt & Interest [2] Difference in effort required to add a new feature in a cleanly designed system, versus a typical system that contains a lot of technical debt. Source: Neal Ford, Evolutionary Architecture and Emergent Design You can think of the inherent complexity as the principal, and the extra effort imposed by previous expedient shortcuts as the interest. 101

×