Refactoring, Emergent Design & Evolutionary Architecture

Refactoring,
Emergent Design &
Evolutionary Architecture
by Brad Appleton
Created July 2009
(last updated, October 2010)
ing
Ref
or
Ref
ing
act
or
act
ing
ing
act
or
act
ing
Ref
or
Ref
ing
or
Ref
ing
act

Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton 2
Presenter Information
Brad Appleton
25+ years in industry – last 20 w/large Telecom.
Agile Leader, Coach & Manager
 Scrum, XP, SAFe, LeSS, DAD
ALM/SCM/DevOps Solution Architect
 Git, JIRA, Confluence, TFS, RTC, Jenkins, CI/CD
Leading adoption & scaling of Agile development for
large organization since 2000.
Co-author: Software Configuration Management
Patterns (Addison-Wesley 2002), Agile CM Environments
(column in CM Journal), Reviewer for The Agile Journal

Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton
Outline
• Part I: Refactoring for Agility
 Refactoring Overview
 How to Refactor
 Refactoring to Patterns & Principles
• Part II: Refactoring @ Scale
 Emergent Design
 Technical Debt
 Restructuring
 Evolutionary Architecture
 Incremental Design
• Part III: Special Topics & Further Info
 Refactoring Legacy Code
 Refactoring Databases & Web Content
 Refactoring Test Code
 Additional Resources (Books & Articles)
3

Part I
Refactoring for Agility
ing
Ref
or
Ref
ing
act
or
act
ing
ing
act
or
act
ing
Ref
or
Ref
ing
or
Ref
ing
act

Outline for Part I
• Overview of Refactoring
 Refactoring Defined
 Refactoring Myths & Misconceptions
 The Practice of Refactoring
 Why Refactor?
• How to Refactor
 Simple Code & Clean Code
 The Refactoring Cycle
 Code Smells & Refactorings
 When to Refactor?
• Refactoring to Patterns & Principles
 Design Attributes / Code Qualities
 Design Principles & Design Patterns
 Other Acronyms of Simple/Agile Design
• Summary & Resources for Part I
5

Overview of Refactoring
• Identifies design-maintenance issues (“code smells”)
that typically represent violations of known principles
of good design.
• Incrementally and iteratively applies a set of design
improvement techniques (“refactorings”).
• The goal is to minimize complexity & duplication in
order to maximize simplicity & ease-of-change.
• Encourages the “right” design details to emerge
“just-in-time” with minimal guesswork/rework.
• Scaling-up includes the use of periodic restructuring,
initial & incremental design (“just enough”), and
evolutionary architecture.
6

Refactoring Defined
Refactoring (noun): a change made to the internal
structure of software to make it easier to understand
and cheaper to modify without changing its
observable behavior.
Refactoring (verb): to restructure software by applying
a series of refactorings without changing its
observable behavior.
— Martin Fowler, Refactoring: Improving the Design of Existing Code
“A change to the system that leaves its behavior
unchanged, but enhances some nonfunctional quality –
simplicity, flexibility, understandability, performance.”
— Kent Beck, Extreme Programming Explained
7

Refactoring Distilled
Refactoring is a disciplined technique for the
restructuring of an existing body of code,
altering its internal structure without
changing its external behavior.
 Its heart is a series of small behavior preserving
transformations.
 Each transformation (called a ‘refactoring’ ) does only a
little, but a sequence of transformations can produce a
significant restructuring.
 Since each refactoring is small, it's less likely to go
wrong.
 The system is kept fully working after each small
refactoring, reducing the chances that a system can
get seriously broken during the restructuring.
Source: Martin Fowler, www.refactoring.com
8

Refactoring Myths
9

Refactoring is NOT …
“Rework” – redesigning things that could, and should, have been
designed correctly in the first place.
Design “gold-plating” – work that adds no business value, and
merely serves to stroke the egos of perfectionists who are out of
touch with business reality.
Miscellaneous code “tidying” – the kind that is “nice to have,” but
should only happen when the team has some slack-time, and is a
luxury we can do without, without any serious consequences.
A license to “hack” – avoiding any and all initial design & analysis
and instead jumping straight to coding with no “real” design.
Reengineering – large-scale restructuring that requires a concerted
effort over the course of several weeks/months to re-write or re-
architect significant parts of the system.
Derived from: Patrick Wilson, Continuous Refactoring and the Cost of Decay
10

Refactoring IS …
A systematic approach to source-code “hygiene” that minimizes
the chances of introducing bugs
Improving the design of the code after it has been written
A behavior-preserving transformation of source-code structure
The process of simplifying & consolidating a work-product by
making several, small, successive revisions focused on:
 preserving correctness,
 removing redundancy,
 revealing thoughts & intentions, and …
 improving clarity & conciseness.
A disciplined way of making changes while exposing the project
to significantly less risk.
An effective means to address the economic reality of software
growth/complexity by reducing & amortizing its cost throughout
the daily business of development & maintenance activities.
11

The Practice of Refactoring
The process of refactoring involves:
 the removal of duplication & dependencies
 the simplification of complex logic/structure,
 and the clarification of unclear code & design intentions
Each step is small and simple, even simplistic.
 You move a field from one class to another, pull some code out of
a method to make into its own method, and push some code up or
down a hierarchy.
To refactor safely, you must run tests [preferably automated]
to ensure your changes didn't break anything.
 You will have more confidence to refactor and make other design
changes if you can quickly run automated tests.
Refactoring in small steps helps prevent introducing defects.
 Most refactorings take only seconds or minutes to perform.
 Even large restructurings are implemented in small steps.
Sources: Martin Fowler, Joshua Kerievsky
12

Why Refactor?
Lots of reasons …
 Make it easier to add new code
 Improve the design of existing code
 Increase readability/understanding of code
 Make coding less annoying/frustrating
 Prevent “design decay”
 Clean-up/Simplify messy code
 Find & fix design-principle violations
 Reduce debugging time
 Incorporate learning about the application
13

How to Refactor
The overarching goal of refactoring is to:
• minimize complexity & duplication in order to
• maximize simplicity & ease-of-change.
When we refactor we are striving to produce
code that is clean and simple.
Clean Code follows “Uncle” Bob Martin’s The
Boy Scout Rule:
 When touching any code, leave the codebase cleaner
than you found it!
Simple Code follows Kent Beck’s Four Rules of
Simple Code …
14

Simple Code
15

The Rules of Simple Code
Simple Code …
1. Passes all the tests
2. Contains no duplication
3. Clearly reveals & expresses the developer’s
intent
4. Minimizes the number and size of entities
 classes/modules
 methods/subroutines
Note: these are in order of highest priority first!
16

Clean Code
17

The Rules of Clean Code
Keeping the Code “Clean” means ...
 We must continuously simplify our code (following the
rules of simple code)
 We must continuously clean-up the code, and not
tolerate messes, spills, or “smells” in our code.
 We must not regress into bad habits.
—Joshua Kerievsky, Introduction to Refactoring to Patterns
• Rules for Clean Code come from the book of the same
name by Robert C. Martin (a.k.a. “Uncle Bob”) et.al.
 Clean Code: A Handbook of Agile Software Craftsmanship
• Also see Uncle Bob’s Clean Code Tip of the Week
series of articles online
18

The Refactoring Cycle
19

The Refactoring Cycle
Check for “Code smells”
Refactor
Re-test
After newly added/changed
code passes its tests, you:
1. Check it for “Code smells”
2. When you find one, apply
the specific refactoring that
resolves it!
3. Re-run automated tests to
ensure nothing broke (fix
any “breakage”
immediately)
4. Repeat until the affected
code is smell-free!
20

Code Smells
21

Code Smells
“Any symptom in the source code of a computer program that
indicates something may be wrong. It generally indicates that
the code should be refactored or the overall design should be
reexamined.”
• A Code Smell is a hint that something might be
wrong, not a certainty.
• Major approaches to programming “hygiene”:
 Pragmatic: code smells should be considered on a case by case
basis
 Purist: all code smells should be avoided, no exceptions
 Possible bad practice to a pragmatist, but a sure sign of bad practice
to a purist
• Beware of Code Stenches
22

Code Smell Examples
• Duplicate code or Duplicated method - a code, method,
function, or procedure that is very similar to another
• Large method or class - has grown too large
• Lazy class - a class that does too little
• Feature envy - a class that uses methods of another class
excessively
• Inappropriate intimacy - a class that has dependencies on
implementation details of another class
• Refused bequest - a class that overrides a method of a base
class in such a way that the contract of the base class is not
honored by derived class
• Contrived Complexity - forced usage of overly complicated
design patterns where simpler design would suffice
• Code not actually ever used
23

Categories of Refactorings
Small Refactorings
 (De)Composing methods
 Moving features between objects
 Organizing data
 Simplifying conditional expressions
 Dealing with generalization
 Simplifying method calls
Larger Refactorings/Restructurings
 Tease apart inheritance
 Extract hierarchy
 Convert procedural design to objects
 Separate domain from presentation
Each category contains as many as a dozen or more refactorings,
most of which are catalogued at http://refactoring.com/catalog/
24

Refactorings
Some refactorings from real projects
 Add a Parameter
 Extract Method
 Inline Method
 Inline Class
 Extract Interface
 Move Field
 Move Method
 Parameterize Method
 Rename Method
 Method Parameter to Constructor Parameter
Composite Refactorings (often to Design Patterns)
See http://refactoring.com/catalog/ for an up-to-date list
(and the “Refactoring to Patterns” catalog too)
25

What to do if …?
I spot a “smell” that is not already known or
catalogued?
 Convince yourself it really is a “smell” by trying to
identify its ill-effects and/or the design principle it
appears to violate
There is no specific known/catalogued
“refactoring” for what I think I need?
 Check again, just to be sure you didn’t miss it
 Consult a co-worker
 Use known design principles and/or software
modifiability tactics to guide your judgment
 Check for known design patterns to resolve that
specific problem for your particular scenario
26

When to Refactor?
 While adding functionality
 While fixing a bug
 While reviewing code
 After coding the same/similar thing for the
third time (to “factor out” the duplication)
A.k.a., The Rule of Three: 3 strikes and you refactor.
 After the third time you deferred refactoring a
change, for any reason [The Rule of Three, again]
 Before the end of the iteration if you haven’t
been following The Rule of Three
27

Refactoring Continually
“It's best to refactor continuously (as a normal
part of each coding activity) rather than in
phases or increments.
 When you come across code that is in need of improvement,
improve it!
 On the other hand, if you must finish a feature before a demo
scheduled for tomorrow, finish the feature and refactor later.
Business is well served by continuous refactoring, yet
refactoring must coexist harmoniously with business
priorities.” ―Joshua Kerievsky, Introduction to Refactoring to Patterns
“In my view, refactoring is not an activity you set aside
time to do separately from implementation activities.
Refactoring is something you do all the time in little
bursts.” ―Martin Fowler, Refactoring: Improving the Design of Existing Code
28

When NOT to Refactor?
 When the build is broken or tests don’t pass
 When it would compromise meeting an
impending deadline or commitment
 When the code in question really just needs to
be re-written “from scratch”
 When it would modify code/interfaces that could
significantly impact/break other work:
 Published/public interfaces and protocols
 Database schemas/tables/operations
Sometimes we must defer refactoring for later
and/or plan for subsequent restructuring
29

Refactoring to Patterns & Principles
Software Design Principles & Design Patterns
are the underlying foundation for Refactoring:
 Code smells (a.k.a “code pathologies”)
 Signal a possible violation of design principles
 Suggest which refactoring may be needed
 Refactorings
 Correct a design principle violation (at least partially)
 Converge toward common design patterns
 Design Patterns
 Reconcile forces among conflicting design concerns
 Restore balance between competing design principles
 Design Principles
 Lead us to attain desired design qualities/attributes
30

Design Attributes/Code Qualities
31

Design Attributes/Code Qualities
Qualities of Highly Maintainable Software:
 Loose Coupling & High Cohesion
 Hierarchy (Structural Decomposition)
 Abstraction, Encapsulation & Modularity
 Sufficiency, Parsimony and Primitiveness
 Readability
 Testability
 Modifiability
 Serviceability
32

Design Principles
33

Design Principles: SOLID, SoC, DRY, Shy
The SOLID Principles of Object-Oriented Design:
SRP : The Single Responsibility Principle
OCP : The Open-Closed Principle
LSP : The Liskov Substitution Principle
ISP : The Interface Segregation Principle
DIP : The Dependency Inversion Principle
The SoC Principle: Separation of Concerns — separate
interface from implementation, policy from mechanism, behavior
from construction, commands from queries, levels of abstraction, ...
The DRY Principle: Don’t Repeat Yourself (Eliminate
Duplication), Single Point of Truth (SPOT)
The “Structure-Shy” Principle: (“Tell, Don’t Ask!”), The
Law of Demeter, Principle of Least Assumed Knowledge
34

Other Acronyms of Simple/Agile Design
OAOO – Say Things Once And Only Once
(restatement of the DRY principle)
DTSTTCPW – Do The Simplest Thing That Could
Possibly Work! (restatement of the KISS principle)
YAGNI – You Aren’t Gonna Need It!
The LRM Principle: Defer Commitment of
Irreversible Decisions to the Last Responsible
Moment!
BDUF – Big Design Up-Front! (vs. JEDI)
JEDI – Just Enough Design Initially/In-front!
DDD – Domain-Driven Design
35

Design Patterns
36

Design Patterns
A Design Pattern is …
• A recurring solution to a common problem in a given context and
system of forces.
• A named "nugget" of instructive insight, conveying the essence of a
proven solution to a recurring problem in a given context amidst
competing concerns.
• A successfully recurring "best practice" that has proven itself in the
"trenches."
• A literary format for capturing the wisdom and experience of expert
designers, and communicating it to novices.
Design patterns …
• Establish a common terminology and professional vocabulary for
discussing recurring design problems & their reusable solutions
• Raise the “level of thinking” in our communication and
understanding about software design
• Are context-specific applications of general design principles
37

Design Patterns Examples
Examples of well-known Design Patterns:
• Creational: Abstract Factory, Builder, Factory Method,
Prototype, Singleton
• Structural: Adapter, Bridge, Composite, Decorator, Façade,
Flyweight, Proxy
• Behavioral: Chain of Responsibility, Command, Interpreter,
Iterator, Mediator, Memento, Observer, State, Strategy, Template
Method, Visitor
Patterns & pattern catalogs are available for:
• Concurrent, Parallel & Distributed Systems, Network Architecture,
User-Interaction Design, Secure Programming, Database Access,
Real-time/Embedded, Information Design, Software Product-Lines,
J2EE (and core J2EE), dotNET, EA, SOA, and Many others
38

Summary: Refactoring for Agility
 Successively applies small behavior-preserving
transformations to eliminate code smells
 Based on proven design principles and patterns for
achieving maintainability & modifiability
 Good automated testing is a prerequisite
 Refactoring is not rewriting, rework or
restructuring
 With refactoring, we continuously invest nominal
effort to reduce the risk & cycle-time of changes
 The goal is to minimize complexity & duplication in
order to maximize simplicity & ease-of-change.
39

Summary: Refactoring for Agility
When practiced in a highly disciplined
manner, refactoring promotes:
 Sufficient functionality
 Simple & clean code
 Supple design
 Serviceable software
 Sustainable team velocity
40

Resources: Code Smells
• Introduction to Code Smells and Refactoring from TestingEducation.org
• Code Smells and Refactoring to Patterns, by Josh Kerievsky
• Refactoring and Code Smells
• The original WikiWiki CodeSmells page
• Martin Fowler's Code Smells page
• A Catalogue of Bad Smells in Code, by Martin Fowler and Kent Beck
• A Taxonomy of Bad Code Smells
• CodingHorror.com's list of Code Smells
• Smells to Refactorings Quick Reference, by Josh Kerievsky
• Gene Garcia's mapping of CodeSmells to Refactorings
• Using Static Analysis Tools to Identify Code Smells, IBM
developerWorks article by Paul Duvall
41

Resources: Design Principles
• S.O.L.I.D. Principles e-Book (free)
http://www.lostechies.com/content/pablo_ebook.aspx
• An O-O Primer, http://www.rgoarchitects.com/Files/ooprimer.pdf
• The Principles of Object-Oriented Design:
http://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod
• Design Principles and Patterns,
http://www.objectmentor.com/resources/articles/Principles_and_Patterns.pdf and
http://www.oodesign.com/
• The Art of SoC (Separation of Concerns), http://www.ctrl-shift-
b.com/2008/01/art-of-separation-of-concerns.html
• Composed Method & Single-Level of Abstraction,
http://www.ibm.com/developerworks/java/library/j-eaed4.html
• Tell, Don’t Ask, http://www.pragprog.com/articles/tell-dont-ask
• O-O in One Sentence: Keep it Shy, DRY, and Tell the Other Guy!
http://media.pragprog.com/articles/may_04_oo1.pdf
42

Resources: Design Patterns
Online Resources:
• Wikipedia on Design Patterns
• The Patterns Home page: hillside.net/patterns
• Design Patterns Reference and Patterns Tutorials
• Patterns & Software: Essential Concepts & Terminology
Books:
• Design Patterns, by the “Gang of Four” (the book that started it all)
• POSA Series of books on Pattern-Oriented Software Architecture
• Head First Design Patterns, from the O’Reilly Head First Series
• Refactoring to Patterns; by Joshua Kerievsky
• Agile Software Development, Principles, Patterns, and Practices
by Robert C. Martin
• Implementation Patterns; by Kent Beck
• Design Patterns Explained (2ed); by Alan Shalloway, James Trott
43

Resources: Other Agile Design Slogans
• XP Simplicity Rules, http://c2.com/cgi/wiki?XpSimplicityRules
• Essential XP: Simple Design,
http://www.xprogramming.com/xpmag/expEmergentDesign.htm
• DTSTTCPW -
http://c2.com/cgi/wiki?DoTheSimplestThingThatCouldPossiblyWork
• OAOO - http://c2.com/cgi/wiki?OnceAndOnlyOnce
• YAGNI - http://en.wikipedia.org/wiki/You_Ain%27t_Gonna_Need_It
• LRM - http://stevesmithblog.com/blog/delaying-decisions/,
http://vig.pearsoned.co.uk/samplechapter/0321150783.pdf
• BDUF - http://en.wikipedia.org/wiki/Big_Design_Up_Front
• JEDI - http://www.featuredrivendevelopment.com/node/507
• DDD - http://www.domaindrivendesign.org/
44

Part II
Refactoring @ Scale
ing
Ref
or
Ref
ing
act
or
act
ing
ing
act
or
act
ing
Ref
or
Ref
ing
or
Ref
ing
act

Outline for Part II
• Scaling-up!
• Emergent Design
• Technical Debt
 Technical Debt – Tips & Truths
 Restructuring Technical Debt
• Restructuring
 Refactoring vs. Restructuring
 Restructure Periodically
 Architecture Smells & Modifiability Tactics
 When to Plan for Restructuring?
• Evolutionary Architecture
 Incremental Design
 JEDI (Just Enough Design Initially)
 Supple Design & Domain-Driven Design
• Resources for Part II
46

Scaling-Up

Scaling-Up
To scale refactoring for larger projects, additional
techniques & concepts must be added to the mix.
 Note that this is “in addition to” (not “instead of”)
Refactoring
In-the-Small
Refactoring @ Scale
(adds the following)
Small, Fast & Frequent
Refactorings
Larger, Periodic & Planned
Restructurings
Emergent Design Incremental Design &
Deferred Refactoring Restructuring & Technical Debt
Code Smells Architecture Smells
Design Principles & Patterns Software Modifiability Tactics
Simple/Clean Code Supple/Domain-Driven Design

Emergent Design
49

Emergent Design
Emergent Design is a fancy name for the resulting
design that “emerges” from the synergy of
combining Refactoring together with TDD,
Continuous Integration and Automated Testing.
“As a result, you’ll find the balance of work changes.
 Design, rather than occurring all up front, occurs continuously
during development.
 The cumulative effect of these small changes can radically
improve the design.
 You learn from building the system how to improve the design.
 The resulting interaction leads to software with a design that stays
good as development continues.
 It is the exact reverse of the normal notion of software decay.”
― Martin Fowler, Refactoring: Improving the Design of Existing Code
A lack/lapse of discipline in any of these technical
practices will result in the accrual of technical debt!
50

Technical Debt
51

Technical Debt [a.k.a. Design Debt]
Technical debt occurs when our software becomes difficult
or risky to change, and takes increasingly more time and
effort to evolve.
• It represents the accumulated amount/cost of rework that
will be necessary to correct and/or recover from the
deviation between:
 the current design of the system versus …
 a design that is minimally complex yet sufficiently complete to
ensure correctness & consistency for timely delivery.
• This effort grows more than linearly over time as a system
becomes bigger and more complex.
• The economic impact of technical debt is directly related
to the cost of complexity and its resulting “friction”
against the velocity of the development team.
52

Technical Debt – Real Costs
The New Push to Measure Software’s True Cost,
from ComputerWorld.com, October 2010:
• The idea that software acquires a "technical debt" that is paid
in real dollars is getting new attention and research:
 The National Science Foundation approved a $465,000 research
grant last year on technical debt
 Gartner Research just released its study on the subject of "IT
debt“, and puts the current IT debt bill at $500 billion worldwide,
and says that it will double in five years to $1 trillion.
 A company that makes software quality tools, Cast, just released
a study gleaned from customer software evaluations that puts
the cost of technical debt at $2.82 per line of code. For the
average-size software application of 374,000 lines of code, this
amounts to just over $1 million in technical debt.
53

Technical Debt – Tips & Truths
• Incurring technical debt means your velocity slows
down and you will deliver less value
• The cost of getting out-of-debt is compounded over
time: the longer you wait, the faster it grows
• If you plan to incur technical debt, the persons
responsible must have a workable plan to pay it off!
• “Interest only” payments won’t improve things
• Pay early, pay often, and pay-as-you-go. (The only
other options are bankruptcy or death.)
• Remember: Those with the worst debt problems often
have the most difficulty imagining a life without borrowing!
derived from http://agileinaflash.blogspot.com/2009/02/technical-debt.html
54

Restructuring Technical Debt
If we accrue a non-trivial amount of technical debt, we
can’t simply “refactor” it away.
 Paying it off typically requires restructuring efforts (or even
reengineering) that must be planned.
 Iteration plans must accommodate specific tasks for these
restructuring efforts (or even be dedicated to restructuring).
 Ignoring it, or deferring it for very long is not a viable option!
“All successful software gets changed. So if we think we’re working on
code that will be successful … we need to keep it easy to change.
Anything that makes code difficult to change is technical debt.
Just like any other debt, the cost of paying off technical debt gets
more and more expensive over time. …
Technical debt drives the total cost of software ownership
relentlessly higher … eventually we will have to pay it off or the
system will go bankrupt.”
―Mary Poppendieck, Leading Lean Software Development
55

Restructuring
56

Overview of Restructuring
• Identifies higher-level issues (“architecture smells”)
that typically represent violations of known principles
of good software architecture & design.
• Periodically applies larger-scale “refactorings” and/or
many small refactorings that were previously deferred.
• The goal is to “pay down technical debt” in order to
limit the increasing costs of accumulated complexity.
• Typically requires a concerted effort that must be
separately planned.
• Uses not only design patterns/principles, but also
architectural patterns/principles, as well as software
modifiability tactics.
57

Refactoring vs. Restructuring
The following are not valid examples of refactoring:
● The system being broken (unbuildable) for a couple of days
while someone is “refactoring”
● Reworking the structure of an entire document or web
site/portal
● A concerted effort by one or more people over several
days/weeks to rework and consolidate the code/design
structure
All of the above are restructuring:
Restructuring is any rearrangement of parts of a whole. It's a
very general term that doesn't imply any particular way of
doing the restructuring.
Refactoring is a very specific technique, founded on using
small behavior-preserving transformations (themselves
called refactorings).
―Martin Fowler, Refactoring Malapropism
58

Restructure Periodically
• Restructuring is often associated with absent or
neglectful refactoring and/or design.
• But … Any large software project spanning multiple
teams eventually needs restructuring.
 Even in the presence expert-level architecture, design &
continuous refactoring
 This is just a reality of software evolution/entropy
• Therefore … Large software projects should assume
that periodic restructuring will be necessary, and
should plan accordingly to:
 Clean-up accumulated code-smells and apply numerous
refactorings that were deferred but are now sorely needed,
 Address architecture smells by applying restructurings, patterns
and modifiability tactics that have broader impact.
59

Architecture Smells
60

Architecture Smells
Smells in Dependency Graphs
• Obsolete Classes
• Tree-like, Static Cycles
• Visibility of Dependency Graphs
Smells in Inheritance
Hierarchies
• Type Queries, List-like Hierarchy
• No Subclass Method Redefinition
• No Polymorphic Assignments
• Parallel or Too Deep Hierarchy
Smells in Packages
• Unused Packages
• Cyclic Package Dependencies
• Packages Too Small / Large
• Package Hierarchies Unbalanced
• Package Hierarchies Not Clearly
Named
Smells in Subsystems
• No Subsystems
• Cycles between Subsystems
• Subsystem Too Large / Small
• Subsystem-API Bypassed
• Subsystem-API Too Large
• Overgeneralization
Smells in Layers
• No Layers, Too Many Layers
• Strict Layers Violated
• Upward References Between
Layers (Cycles between Layers)
• Inheritance between Protocol-
Oriented Layers
• References between Vertically
Separated (non-adjacent) Layers
Source: Refactoring in Large Software Projects, by Martin Lippert & Stefan Roock
61

Software Modifiability Tactics
62

Software Modifiability Tactics
Localize Changes (increase cohesion)
 Maintain Semantic Coherence
 Anticipate Expected [types of] Changes
 Generalize the Module
 Limit Possible Options
 Abstract Common Services
Prevent Ripple Effects (reduce coupling)
 Hide Information
 Maintain Existing Interfaces
 Restrict Communication Paths
 Use an Intermediary
Defer Binding-time (defer decision-making)
 Run-time Registration
 Configuration Files
 Polymorphism/Delegation
 Component Replacement
 Adhere to Defined Protocols Source: SEI Technical Report CMU/SEI-2007-TR-002
63

When to Consider/Plan Restructuring?
Consider whether restructuring effort is needed …
• During Planning (of course)
 Release planning / Sprint planning
 Of “special” iterations (e.g., “spikes”, architecture, etc.)
 Of stabilizing (“hardening”) iterations
• During Retrospectives
 Iteration retrospectives
 Inter-team (“joint”) retrospectives [Scrum-of-Scrums level]
• After Reviews
 After a Sprint review
 After a Technical/Release/Program review
• After three iterations without any restructuring
64

65

Software Architecture concerns infrastructure elements
that must exist before you can begin execution.
 Architecture is about things that are hard to change later, it is
difficult to allow an architecture to emerge.
 For large projects, this includes high-level organization of the system
into functionality/elements that will be allocated to separate teams.
However, just because we can't allow architecture to emerge
doesn't mean that it can't evolve.
 If we create an initial, flexible architecture and take special care to
not create an irreversible decision, then we can allow it to evolve
over time as new concerns appear.
— Neal Ford, Evolutionary Architecture and Emergent Design
Key techniques of Evolutionary Architecture include:
 Deferring Irreversible Decisions to the “Last Responsible Moment”
(LRM Principle)
 Architectural “Spike” (a.k.a. Architectural “Deep Dive”)
 Architecture Iteration and/or Spike Iteration
66

Incremental Design
Design evolves incrementally, iteration by iteration,
based on current business priorities and discovered
technical limitations.
“Invest in the design only what is needed to comfortably support the current
iteration.” —Kent Beck & Cynthia Andres
Incremental Design …
• Does not prohibit thinking about higher-level design.
• Does encourage planning in detail only what will be
constructed soon.
• Focuses on “Just Enough, Just-In-Time” :
 Specifying too much detail too soon causes more rework later.
 But doing less now and saving the rest for later should not require
significantly more work later than it would today.
 We must do Just Enough Design Initially to attain the right balance
of anticipation and adaptation.
67

Just Enough Design Initially (JEDI)
68

Just Enough Design Initially (JEDI)
Initial design (before coding) is still necessary.
 This use of “JEDI” was coined by Stephen Palmer as part of
Feature-Driven Development (FDD)
Basic rule of thumb to tell when “JEDI” is achieved:
 At iteration-scope, when, after one pass through the iteration
backlog, modeling in small groups does not produce any new
classes or associations of real significance.
 At task/TDD scope, when we have defined enough structure &
interface(s) to know specifically what code to write/test, precisely
where to write it, and exactly how to invoke it.
Techniques of “the JEDI way” include:
 Collaborative Domain Modeling and Color Modeling [from FDD]
 Supple Design techniques & patterns
 Domain-Driven Design (a.k.a. DDD -- see domaindrivendesign.org)
 Design Blitz & other Agile Modeling techniques (see
agilemodeling.com)
69

Supple Design
70

Supple Design & DDD
Domain-Driven Design (DDD) approaches modeling the
core logic of the software by focusing on the domain.
 The basic idea is the design should directly reflect the core business
domain and domain-logic of the problem to solve
 This helps understanding the problem as well as the implementation
and increases maintainability of the software.
DDD uses common principles and patterns as "building
blocks" to model & create a “supple design”
 Supple: pliant, malleable, limber, yielding or changing readily.
 The design is firm yet flexible, with structure and intent both clearly
conveyed and deeply realized by the code.
Patterns of Supple Design include:
 Intention-Revealing Interfaces, Ubiquitous Language, Side-Effect-
Free Functions, Assertions, Conceptual Contours, Standalone
Classes, Closure of Operations, Declarative Style
71

Resources: Evolutionary Architecture
• Neal Ford's series of IBM developerWorks articles on Evolutionary
Architecture and Emergent Design
• Chris Sterling's book on Architecture in an Agile Organization
• Presentations by Ryan Shriver at theagileengineer.com
• Dean Leffingwell's writings on Agile Architecture
• OOPSLA '09 workshop on "Architecture in an Agile World"
• Software Architecture and Agile Software Development - An Oxymoron? by
Philippe Kruchten
• Agile Architecture - How much is enough?, by Eoin Woods
• The Agile Architect site (including the role of the agile architect)
• Agile Architecting by Erik Philippus
• Scott Ambler's Scaling Agile Development via Architecture
• Lean Software Architecture, Jim Coplien & Gertrud Bjornvig
• Architecture Meets Agility, by Hakan Erdogmus
• Systems Engineering and Architecting Challenges: Application to Agile
Development, by Murray Cantor
72

Resources: Emergent Design and
Incremental Design
• Emergent Design: The Evolutionary Nature of
Professional Software Development, by Scott L. Bain
• Neal Ford, Evolutionary Architecture and Emergent
Design series of articles on IBM developerWorks
• Is Design Dead? by Martin Fowler
• The Mikado Method, by Daniel Brolund & Ola Ellnestam
73

Resources: Restructuring
• InfoQ: Michael Stal on Architecture Refactoring
• Refactoring in Large Software Projects: Performing
Complex Restructurings Successfully, by Martin Lippert,
Stephen Roock (also an earlier version online)
• The Mikado Method and Refactoring Large Software
Systems
• Object-oriented Reengineering Patterns, by S. Demeyer, S.
Ducasse & O. Nierstrasz (freely downloadable online)
74

Resources: Technical Debt
• Technical Debt and Design Death, by Kane Mar and Michael James
• Design debt economics, by John Elm, IBM developerWorks, June 2009
• The Agile Executive articles on Technical Debt, by Israel Gat
• Managing Technical Debt and 10X Software Development - Technical Debt,
by Steve McConnell
• Managing Technical Debt; by Tom Brazier
• Repaying Technical Debt; by Simon Baker
• Software Debt and Technical Debt, by Chris Sterling, author of Managing
Software Debt (also see video presentation)
• Software Entropy: Don’t Tolerate Broken Windows, and Zero-Tolerance
Construction by Andy Hunt and Dave Thomas
• Monetizing Technical Debt, and other InfoQ articles on technical debt
• Technical Debt - How not to ignore it! by Henrik Kniberg
• Continuous Refactoring and the Cost of Decay, by Patrick W. Welsh
75

Resources: Modifiability Tactics
• Modifiability Tactics, SEI @ CMU Technical Report CMU/SEI-
2007-TR-002
 http://www.sei.cmu.edu/publications/documents/07.reports/07tr002.html
• Modifiability Tactics, Chapter 5 section 3 of Software
Architecture in Practice
 http://www.tar.hu/softarchpract/ch05lev1sec3.html
• Understanding Architectural Patterns in Terms of Tactics
and Models
 http://www.sei.cmu.edu/news-at-sei/columns/the_architect/2007/08/architect-2007-08.htm
• Design and Tactics, Lecture notes from an Aalborg University
course on Database and Software Architecture
 http://www.cs.aau.dk/%7Emly/dbsa07/L8_Design_and_Tactics.pdf
76

Resources: Supple Design & DDD
• Wikipedia page on DDD: http://en.wikipedia.org/wiki/Domain-driven_design
• DDD Intro Article: http://www.infoq.com/articles/ddd-in-practice and http://www.typo3-
media.com/blog/domain-driven-design-introduction.html
• DDD – a Brief Introduction:
http://www.typo3-media.com/blog/domain-driven-design-introduction.html
• DDD: Supple Design Patterns
http://www.cs.colorado.edu/~kena/classes/6448/s05/lectures/lecture30.pdf
• DDD Pattern Summaries:
http://domaindrivendesign.org/resources/what_is_ddd
• Free mini-eBook: Domain-Driven Design Quickly
http://www.infoq.com/minibooks/domain-driven-design-quickly
• Yet another free mini-eBook: Step-by-Step Guide to DDD
http://dddstepbystep.com/
• See also http://domaindrivendesign.org/ and http://www.infoq.com/domain-driven-design
77

Part III
Special Topics and Further
Information/Resources
ing
Ref
or
Ref
ing
act
or
act
ing
ing
act
or
act
ing
Ref
or
Ref
ing
or
Ref
ing
act

Refactoring & Legacy Code
From Michael Feathers’ book Working Effectively with Legacy
Code (WELC) :
 Legacy Code is defined as any existing code that does not have
adequate automated test coverage.
Recommends an incremental approach for legacy code:
 Refactor & automate tests for legacy methods/subroutines whenever
you are already “in the neighborhood” to modify them for an existing
task in the current iteration.
 Don’t attempt a comprehensive & thorough “legacy clean-up” all at
once (no “magnum opus” grand-redesign-in-the-sky)
 Break-up larger/significant “legacy clean-up” efforts into smaller
chunks and defer to the next planned restructuring.
79

Refactoring Databases
See the following:
• Refactoring Databases: Evolutionary Database Design, by Scott
W. Ambler & Pramodkumar J. Sadalage
• Refactoring SQL Applications, by Stephane Faroult, Pascal
L'Hermite
• Catalog of database Refactorings, by Scott Ambler et.al.
• Evolutionary Database Design, by Martin Fowler
• Agile Database Refactoring with Hibernate, by Gilad Buzi, Kelley
Glenn, and Jonathan Novich.
80

Refactoring Web Content and Test
Code
See the following:
• Refactoring HTML: Improving the Design of Existing
Web Applications, by Elliotte Rusty Harold
• xUnit Test Patterns: Refactoring Test Code, by Gerard
Meszaros
81

Sample Test Code Refactorings
Minimize Data:
 Remove things from the fixture until we have a Minimal Fixture.
Inline Resource:
 Move contents of an external resource into fixture setup logic of the test.
Make Resource Unique:
 Make the name of any resources used by a test unique.
Replace Dependency with Test Double:
 Break dependency by replacing a depended-on component with a Test
Double.
Extract Testable Component:
 Extract logic to test into a separate component that is designed for
testability and is independent of the context in which it is run.
Setup External Resource:
 Create an external resource within the fixture setup logic of the test rather
than using a predefined resource.
Source: www.xunitpatterns.com
82

Online Resources
• The Refactoring homepage - www.refactoring.com
• Refactoring to Patterns Catalog
• Refactoring: Small Steps Guaranteed to Help You Clean Up Your Code,
by C. Keith Ray.
• Introductory Refactoring Articles
• Podcasts, “Refactoring: Part 1 and Part 2,” a talk with Martin Lippert.
• Refactoring Checklist, by Michael Nielsen.
• Michael Stal on Architecture Refactoring, by Niclas Nilsson.
• Introduction to Refactoring to Patterns, by Joshua Kerievsky.
• Code Refactoring Guidelines, by Federico Cargnelutti.
• Smells to Refactorings table, from wiki.java.net
• Refactoring with Code Metrics, Andrew Glover
• Refactoring Thumbnails, from www.refactoring.be
• Continual Refactoring, by Paul Duvall
• Refactoring blogs, books, forums, sample chapters, videos, tools and
tutorials.
83

Books
Refactoring: Improving The Design of Existing Code; by
Martin Fowler
Refactoring to Patterns; by Joshua Kerievsky
Emergent Design: The Evolutionary Nature of Professional
Software Development, by Scott L. Bain
Clean Code: A Handbook of Agile Software Craftsmanship;
by Robert C. Martin
Working Effectively with Legacy Code; by Michael Feathers
xUnit Test Patterns: Refactoring Test Code, by Gerard
Meszaros
Domain-Driven Design: Tackling Complexity in the Heart of
Software; by Eric Evans
Implementation Patterns; by Kent Beck
84

Books
85

Backup Slides
[for reference material]

What Motivates us to Refactor?
• Make it easier to add new code.
 When adding new functionality to a system, we have a choice:
a)implement without regard to how well it fits the existing design,
b)modify existing design to easily accommodate the new feature.
 With a), we incur design debt, which can be paid down later by
refactoring [or more significant restructuring].
 With b), we analyze what must change to best accommodate the
new feature and then make the necessary changes.
• Improve the design of existing code.
 By continuously improving the design of code, we make it easier
and easier to work with.
 Continuous refactoring involves constantly sniffing for code smells
and removing them immediately (or soon after)
 If you get into the hygienic habit of refactoring continuously, you'll
find that it is easier to extend and maintain code.
Source: J. Kerievsky, Introduction to Refactoring to Patterns
87

What Motivates us to Refactor? [2]
• Gain a better understanding of code.
 Sometimes we look at code and have no idea what it does or
how it works.
 Even if someone could stand next to us and explain the code,
the next person to look at it could also be totally confused.
 If the code isn't clear, it's an odor that needs to be removed
by refactoring, not by “deodorizing” the code with a
comment.
• Make coding less annoying/frustrating.
 Sure, we can say we refactor to remove duplication, to
simplify or clarify the code. But what actually compels us to
refactor?
 Sometimes it is emotions. Face it – it’s just plain frustrating to
have to fix or extend code that is poorly structured, overly
complex, and difficult to understand/navigate.
88

Refactoring and Evolution
Working incrementally changes “I wish I had thought of
this sooner” to “Now I get it; let's do it this way.”
It is a fundamentally stronger position to be in.
• Evolution requires change ― refactoring is a disciplined way of
making change while exposing the project to significantly less
risk.
• It allows us to introduce design at a later period, especially
through the concept of refactoring to open-closed.
• Knowing you can do this reduces the ratio of design to over-
design.
— Scott L. Bain Emergent Design: The Evolutionary Nature of Professional Software
Development, May 2008
89

Test-Driven Refactoring
Testing also plays an altogether different role in
refactoring;
 it can be used to rewrite and replace old code. A test-driven
refactoring involves applying test-driven development to
produce replacement code and then swap out old code for
new code (while retaining and rerunning the old code's
tests).
 Just remember that the "reimplement and replace"
technique, as performed by using test-driven refactoring, is
another useful way to refactor.
 While it tends to be most helpful when you're designing a
new algorithm or mechanism, it may also provide an easier
path than applying low-level or composite refactorings.
90

7 Deadly Symptoms of Rotting Design
1. Rigidity – make it hard to change
2. Fragility – make it easy to break
3. Immobility – make it hard to reuse
4. Viscosity – make it hard to do the right thing
5. Needless Complexity – over design
6. Needless Repetition – error prone
7. Not doing any
Source: Robert C. Martin, http://www.rgoarchitects.com/Files/ooprimer.pdf &
http://www.objectmentor.com/resources/articles/Principles_and_Patterns.pdf
91

The Laws of Software Evolution
A series of 8 laws formulated by Lehman &
Belady starting in 1974, including:
 The Law of Continuing Change
 The Law of Increasing Complexity
 The Law of Continuing Growth
 The Law of Declining Quality
The laws can be summarized as follows:
 A system that is being used undergoes continuing
change or degrades in effectiveness.
 A computer program that is changed, becomes less
and less structured. The changes increase the entropy
and complexity of the program.
Source: Wikipedia on “Lehman’s Laws of Software Evolution”
92

Dimensions of software complexity
Higher technical complexity
- Embedded, real-time, distributed, fault-tolerant
- Custom, unprecedented, architecture reengineering
- High performance
Lower technical complexity
- Mostly 4GL, or component-based
- Application reengineering
- Interactive performance
Higher
management
complexity
- Large scale
- Contractual
- Many stake holders
- “Projects”
Lower
management
complexity
- Small scale
- Informal
- Single stakeholder
- “Products”
Defense
MIS System
Defense
Weapon SystemTelecom
Switch
CASE Tool
National Air Traffic
Control System
Enterprise IS
(Family of IS
Applications)
Commercial
Compiler
Business
Spreadsheet
IS Application
Distributed Objects
(Order Entry)
Small Scientific
Simulation
Large-Scale
Organization/Entity
Simulation
An average software project
- 5-10 people
- 3-9 month duration
- 3-5 external interfaces
- Some unknowns & risks
Embedded
Automotive
Software
IS Application
GUI/RDB
(Order Entry)
Source: Walker Royce
93

Managing Software Complexity
In No Silver Bullet, Fred Brooks defines two
categories of complexity:
 Essential complexity: This is the inherent complexity of
the problem domain. There is no avoiding it. (It comes
with the territory.)
 Accidental complexity: This is the complexity of the
solution. It is a byproduct of the systems, languages,
and frameworks we use. (“We have met the enemy, and it is
us!”)
• In principle, accidental complexity can be
reduced by changing the system.
 Refactoring reduces accidental complexity!
94

Accidental Complexity
Three things tend to spawn accidental complexity.
• Design debt -- just-in-time hacks to code because of schedule or
other external pressures.
• Duplication, violations of the DRY (Don't Repeat Yourself)
principle.
 Duplication is the single most insidious diminishing force in software
development because it manages to creep into so many places without
developers even realizing it..
 Duplication harms projects because it resists attempts to make structural
changes or refactor toward better code.
• Irreversibility. Any decision you make that cannot be reversed
will eventually lead to some level of accidental complexity.
 Irreversibility affects both architecture and design, although its effects
are both more common and more damaging at the architectural level.
Try to avoid decisions impossible or cumbersome to reverse.
Source: Neal Ford, Evolutionary Architecture and Emergent Design
95

Why do we call it “Debt”?
Ward Cunningham's financial metaphor of design debt
works far more effectively than technical language
(e.g., “refactoring”) to influence management.
Due to ignorance or the urge to "not fix what ain't
broke" too many developers & teams spend too little
time paying down design debt.
 if you don't pay off a debt, you incur late fees.
 If you don't pay your late fees, you incur higher late fees.
 The more you don't pay, the worse your fees and payments
become.
 Compound interest kicks in, and as time goes on, getting out of
debt becomes an impossible dream.
The same is true with design debt.
Source: Joshua Kerievsky, Introduction to Refactoring to Patterns
96

What Causes Technical Debt?
It is a difficult, delicate and dynamic balancing act to achieve
the necessary and sufficient amount of design to implement
only the essential complexity required by system.
We lack perfect knowledge/certainty, and can’t always get it
right the first time we try:
 Sometimes we knowingly (and under pressure) do something the "quick &
dirty" way, with the intent to “clean it up” later.
 Sometimes we attempt too much to soon, delving into details of the
requirements when we and our customers/users haven't yet learned
enough about the true needs of the system.
 Sometimes we unknowingly violate design principles, resulting in
undesirable dependencies that make code or other work-products hard to
change.
 Sometimes we neglect to properly "tend" to the design and don't give it the
necessary amount of ongoing care and feeding needed to keep it "fit" and
"supple."
97

Principal & Interest
• Every developer becomes aware of the concept of
technical debt, whereby you make compromises in
your design for the sake of some external force, such
as schedule pressure.
 Technical debt resembles credit card debt: you don't have
enough funds at the moment, so you borrow against the future.
 Similarly, your project doesn't have enough time to do something
right, so you hack a just-in-time solution and hope to use some
future time to come back and retrofit it.
• Unfortunately, many managers don't seem to
understand technical debt, causing resistance to
revisiting past work.
98

Software Entropy & Complexity
Building software isn't like digging a ditch.
 If you make compromises when you dig a ditch, you just get
uneven width or unequal depth. Today's flawed ditch doesn't
prevent you from digging a good ditch tomorrow.
 But the software you build today is the foundation for what you
build tomorrow. Compromises made now for the sake of
expediency cause entropy to build up in your software.
Entropy is a measure of complexity, and if you
add complexity now because of a just-in-time
solution to a problem, you must pay some
price for that for the remaining life of the
project.
99

Technical Debt & Interest
• Let's say that you want to add new features to an
existing, long-running project.
 These new features have a certain inherent complexity to them.
 However, if you have technical debt already, you must work
around those compromised parts of the system to add new
features.
• Thus, the cost for additions mirrors the financial
metaphor.
• Figure 2 shows the difference between the effort
required to add a new feature in a cleanly designed
system (for example, one with little or no technical
debt), versus a typical system that contains a lot of
technical debt.
100

Technical Debt & Interest [2]
Difference in effort required to add a new feature in a cleanly designed
system, versus a typical system that contains a lot of technical debt.
You can think of the inherent complexity as the principal, and the extra
effort imposed by previous expedient shortcuts as the interest.
101

Refactoring, Emergent Design & Evolutionary Architecture

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Refactoring, Emergent Design & Evolutionary Architecture

Similar to Refactoring, Emergent Design & Evolutionary Architecture (20)

More from Brad Appleton

More from Brad Appleton (11)

Recently uploaded

Recently uploaded (20)

Refactoring, Emergent Design & Evolutionary Architecture