Restructuring
Improving the modularity of an existing code-base
                  Chris Chedgey
               www.structure101.com
Structure101.com - People
•   Chris Chedgey, Dunmore East, Ireland
•   Paul Hickey, Toulouse, France
•   Paul O’Reilly, Enniscorthy, Ireland
•   Raghvendra Sahu, Delhi, India
•   + 3 contractors
•   No VC’s, no suits
Structure101.com - Products
•   Restructure101
      •   Fix architecture
•   Structure101
      •   Define it
•   Structure101 Build
      •   Enforce it
•   Java, .NET, C/C++, Actionscript, php, …
•   1,000’s users in 100’s companies
Topics

•   What is “Restructuring”?
•   Motivation
•   Understanding structure
•   Restructuring strategies
•   Examples
•   Keeping it structured
What is restructuring?
Refactoring                           Restructuring
•   “Changing code without            •   “Reorganizing a code-base
    modifying behavior to improve         without modifying the code to
    nonfunctional attributes.”            improve modularity”
•   Code is readable                  •   Code-base is understandable
•   A lot of invasive code editing    •   Minimal invasive code editing
•   Scope: small worlds of a few      •   Scope: whole code base; what
    classes at a time; what you see       you don’t see in the IDE
    in the IDE.
What is structure?

•   Composition
•   Dependencies that roll up through composition
•   Nested “levels”
•   Quality factors
      •   Complexity
      •   Modularity
Why is it important?
•   Determines cost of development
•   Predictable impact
•   Testability
•   Reusability
•   Value of your code base
      •   Asset or liability?
Structural quality: complexity
•   Fat                                  •   Eliminate tangles by putting
                                             everything in one place -
      •   Too much complexity in one         creates fat
          place (method, class,
          package, bundle, …)            •   Eliminate fat by simplistically
•   Tangles                                  partitioning fat items – creates
                                             tangles
      •   Cyclic dependencies
      •   Cumulative Component
          Dependency (CCD, Lakos)        Checked
          goes through the roof          automatically
      •   Acyclic Dependency Principle
          (ADP, Bob Martin) – morning
          after syndrome
Structural quality: modularity
•   High cohesion          •   S/W-eng 101
•   Low coupling           •   We ALL know about
                               modularity!
•   Information hiding
•   Interfaces
                           Checked
•   Clear responsibility
                           by human architect
•   …
Visualizing structure – core features
•   Composition
•   Dependency
•   Flow of dependencies
•   Whole codebase
•   Filtering
•   Summarizing
•   Comprehension
Visualizing structure: approaches

•    Lists and treeview
•    Directed graph
•    Dependency Structure Matrix (DSM)
•    Levelized Structure Map (LSM)
Visualizing structure - list & treeview
•   Good to discover specific
    information
•   No dependency flow,
    manipulation
Visualizing structure: directed graphs
•   Info points: #items + #deps
•   Levelizable – good flow
•   #deps becomes a problem
      •   Indicate fat, not fix

•   Infinite nesting not feasible
•   Local comprehension, not for
    global/manipulation
Visualizing structure: DSM
•   Info points: #items2
•   “Scalable” - can be rendered for
    v large graphs
•   Infinite nesting possible
•   Whole code-base
•   Manipulation possible
•   Grow v large
•   Hard to read, bad for the neck
Visualizing structure - LSM
•   Info points: #items
•   Composition, dependency,
    flow, whole codebase,
    filtering, summarizing
•   Goldilocks – designed
    specifically for restructuring
The modularity wasteland
•   Methods ok
•   Classes ok
•   Handfuls of classes ok
       •   Overlapping “small worlds”
       •   Plenty of guidelines
       •   Smells and refactoring
•   But almost no logical organization
    of classes into higher level
    modules
•   This is not scalable
“But we use Java packages!”
•   ` !!
•   Hierarchical ok
•   Support visibility ok
•   BUT: used like a filesystem, not as an
    embodiment of module hierarchy
        •   Composition only

        •   No consideration of dependencies

        •   Little consideration of visibility

        •   Too easy to change (without rules/tools)

•   Becomes a big tangled mess
Modularity stops at class level?
•   Finite aggregation is not
    scalable
•   Oceans of classes
•   Still need to divide and
    conquer
•   Even more – affects the
    whole team
•   Is it inevitable?
Inevitable?
•   No!
•   Spring:
Restructuring strategies
•   Merge parallel structure      •   Do as much as possible by
                                      moving classes and packages
•   Top-down or bottom-up
    decision                      •   Divide disjoint packages early
•   Tackle complexity before      •   Break up v. fat classes and
    modularity                        packages early
•   Break v large class tangles   •   Disentangle
    early
                                  •   Consider mixed granularity
•   Address tangles top-down
                                  •   …
Merge parallel structures
•   Common to separate interfaces/api
    from implementation
•   Perhaps attempt to increase
    awareness of visibility
•   Obscures module structure esp if
    higher in hierarchy
•   Better to consider module sub-items
    as siblings with visibility
•   Keep both if there is a strong
    need/desire for physical code to be
    separate
•   Use of transformations
Merge parallel structures
Bust v large class tangles early
•   Often there is 1 or a few v.large        •   Can the class tangle be contained
    tangles spanning many packages               within a single package?
       •   Making those packages acyclic            •   At least for now?
           means busting the class tangle
                                                    •   Not if classes span to much of the
•   Does it span multiple higher-level                  layering
    layers?
                                                    •   Not if it represents a significant
       •   Break the feedback dependencies              portion of your code base
           between the layers
                                             •   Can you break the tangle into several
                                                 acyclic subsets of tangles?
                                                    •   Each subset can be contained within
                                                        a single package
                                             •   Each subset can be contained within a
                                                 narrow range of ideal leveling
Do as much as you can by moving
classes/packages first
•   Low effort refactoring
•   Less invasive surgery so less risk of impacting behavior
•   Biggest bang for the buck, unless there are v.large class tangles
•   Option to manage architecture separate from physical code
      •   Map classes to physical components
      •   Can’t be done if model requires invasive class editing
Top-down or bottom-up?
•   Top-down                                     •   A bit of both
      •   Preserves as much of the package              •   Preserve top-level breakout
          hierarchy as possible
                                                              •   May reflect team structure
      •   Team can still find classes
                                                              •   Often reflects initial architecture
      •   Feasible if the target structure is
                                                                     •   Sub-modules were not explicitly
          close to the current                                           designed

•   Bottom-up                                           •   Preserve in regions where there is
                                                            relatively small amount of
      •   Current packaging is of little use                restructuring
      •   Target structure is far from current
      •   Often easiest to implement
Make items only using or used by a tangle
Move only used/using child items
Split disjoint packages
Split fat packages and classes
Move feeback from/top
Move feedback to/bottom
Move tangles together
Tackle complexity before (other)
modularity
•   A structure without tangles is much easier to manipulate
•   A milestone in its own right
•   A code-base that is much easier to understand
•   Many simply focus on keeping (over-)complexity out
•   Good basis for further modularization (coupling, cohesion,
    interfaces, …)
Summary
•   The modularity wasteland is normal
•   It costs
•   It can be salvaged
•   It is not easy, but the returns can be huge
•   Questions?
•   Thank you!

2012 02-modularity-animated-120309090811-phpapp02

  • 1.
    Restructuring Improving the modularityof an existing code-base Chris Chedgey www.structure101.com
  • 2.
    Structure101.com - People • Chris Chedgey, Dunmore East, Ireland • Paul Hickey, Toulouse, France • Paul O’Reilly, Enniscorthy, Ireland • Raghvendra Sahu, Delhi, India • + 3 contractors • No VC’s, no suits
  • 3.
    Structure101.com - Products • Restructure101 • Fix architecture • Structure101 • Define it • Structure101 Build • Enforce it • Java, .NET, C/C++, Actionscript, php, … • 1,000’s users in 100’s companies
  • 4.
    Topics • What is “Restructuring”? • Motivation • Understanding structure • Restructuring strategies • Examples • Keeping it structured
  • 5.
    What is restructuring? Refactoring Restructuring • “Changing code without • “Reorganizing a code-base modifying behavior to improve without modifying the code to nonfunctional attributes.” improve modularity” • Code is readable • Code-base is understandable • A lot of invasive code editing • Minimal invasive code editing • Scope: small worlds of a few • Scope: whole code base; what classes at a time; what you see you don’t see in the IDE in the IDE.
  • 6.
    What is structure? • Composition • Dependencies that roll up through composition • Nested “levels” • Quality factors • Complexity • Modularity
  • 7.
    Why is itimportant? • Determines cost of development • Predictable impact • Testability • Reusability • Value of your code base • Asset or liability?
  • 8.
    Structural quality: complexity • Fat • Eliminate tangles by putting everything in one place - • Too much complexity in one creates fat place (method, class, package, bundle, …) • Eliminate fat by simplistically • Tangles partitioning fat items – creates tangles • Cyclic dependencies • Cumulative Component Dependency (CCD, Lakos) Checked goes through the roof automatically • Acyclic Dependency Principle (ADP, Bob Martin) – morning after syndrome
  • 9.
    Structural quality: modularity • High cohesion • S/W-eng 101 • Low coupling • We ALL know about modularity! • Information hiding • Interfaces Checked • Clear responsibility by human architect • …
  • 10.
    Visualizing structure –core features • Composition • Dependency • Flow of dependencies • Whole codebase • Filtering • Summarizing • Comprehension
  • 11.
    Visualizing structure: approaches • Lists and treeview • Directed graph • Dependency Structure Matrix (DSM) • Levelized Structure Map (LSM)
  • 12.
    Visualizing structure -list & treeview • Good to discover specific information • No dependency flow, manipulation
  • 13.
    Visualizing structure: directedgraphs • Info points: #items + #deps • Levelizable – good flow • #deps becomes a problem • Indicate fat, not fix • Infinite nesting not feasible • Local comprehension, not for global/manipulation
  • 14.
    Visualizing structure: DSM • Info points: #items2 • “Scalable” - can be rendered for v large graphs • Infinite nesting possible • Whole code-base • Manipulation possible • Grow v large • Hard to read, bad for the neck
  • 15.
    Visualizing structure -LSM • Info points: #items • Composition, dependency, flow, whole codebase, filtering, summarizing • Goldilocks – designed specifically for restructuring
  • 16.
    The modularity wasteland • Methods ok • Classes ok • Handfuls of classes ok • Overlapping “small worlds” • Plenty of guidelines • Smells and refactoring • But almost no logical organization of classes into higher level modules • This is not scalable
  • 17.
    “But we useJava packages!” • ` !! • Hierarchical ok • Support visibility ok • BUT: used like a filesystem, not as an embodiment of module hierarchy • Composition only • No consideration of dependencies • Little consideration of visibility • Too easy to change (without rules/tools) • Becomes a big tangled mess
  • 18.
    Modularity stops atclass level? • Finite aggregation is not scalable • Oceans of classes • Still need to divide and conquer • Even more – affects the whole team • Is it inevitable?
  • 19.
    Inevitable? • No! • Spring:
  • 20.
    Restructuring strategies • Merge parallel structure • Do as much as possible by moving classes and packages • Top-down or bottom-up decision • Divide disjoint packages early • Tackle complexity before • Break up v. fat classes and modularity packages early • Break v large class tangles • Disentangle early • Consider mixed granularity • Address tangles top-down • …
  • 21.
    Merge parallel structures • Common to separate interfaces/api from implementation • Perhaps attempt to increase awareness of visibility • Obscures module structure esp if higher in hierarchy • Better to consider module sub-items as siblings with visibility • Keep both if there is a strong need/desire for physical code to be separate • Use of transformations
  • 22.
  • 23.
    Bust v largeclass tangles early • Often there is 1 or a few v.large • Can the class tangle be contained tangles spanning many packages within a single package? • Making those packages acyclic • At least for now? means busting the class tangle • Not if classes span to much of the • Does it span multiple higher-level layering layers? • Not if it represents a significant • Break the feedback dependencies portion of your code base between the layers • Can you break the tangle into several acyclic subsets of tangles? • Each subset can be contained within a single package • Each subset can be contained within a narrow range of ideal leveling
  • 24.
    Do as muchas you can by moving classes/packages first • Low effort refactoring • Less invasive surgery so less risk of impacting behavior • Biggest bang for the buck, unless there are v.large class tangles • Option to manage architecture separate from physical code • Map classes to physical components • Can’t be done if model requires invasive class editing
  • 25.
    Top-down or bottom-up? • Top-down • A bit of both • Preserves as much of the package • Preserve top-level breakout hierarchy as possible • May reflect team structure • Team can still find classes • Often reflects initial architecture • Feasible if the target structure is • Sub-modules were not explicitly close to the current designed • Bottom-up • Preserve in regions where there is relatively small amount of • Current packaging is of little use restructuring • Target structure is far from current • Often easiest to implement
  • 26.
    Make items onlyusing or used by a tangle
  • 27.
  • 28.
  • 29.
    Split fat packagesand classes
  • 30.
  • 31.
  • 32.
  • 33.
    Tackle complexity before(other) modularity • A structure without tangles is much easier to manipulate • A milestone in its own right • A code-base that is much easier to understand • Many simply focus on keeping (over-)complexity out • Good basis for further modularization (coupling, cohesion, interfaces, …)
  • 34.
    Summary • The modularity wasteland is normal • It costs • It can be salvaged • It is not easy, but the returns can be huge • Questions? • Thank you!

Editor's Notes

  • #2 Guten abendIch bin ein berlinerDarmstadterGood few years agoBit for freedomLast bit of useless info We dev tools – talk not about these – talk after formal pres
  • #3 6 components to the company, 4 human, 2 beautifulWe don’t own suitsLove what we doRelentless innovation to solve a big problem
  • #5 What – relate to refactoringMotivation – modularity/scalabilityUnderstanding – quantifying over complexityRestructuring strategies – emerging from experience, dozens of client engagements and use casesShow parts of products to illustrate principles, not a tutorial on the products
  • #6 Don’t care whether you use inheritance or aggregationThink of a mass of lines of code, currently organized in one way into methods, classes, packages – we want to reorganize the same lines of code into a different hierarchy of methods, classes, packages.
  • #7 Can you test a tangle of hundreds of classes?Can’t reuse subsets if everything uses everythingWhen a code-base becomes a liability, sw-engineering has failed
  • #9 Structural complexity can be assigned thresholds and checked objectively/automatically
  • #10 We learn about modularity in SW-eng101Builds on complexity principles
  • #15 Another representation of a graphManipulation involves dragging rows and columns around