Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tool Driven Restructuring of Large Codebases

198 views

Published on

Presented in s/w architects meetup on 13th April.

Published in: Software
  • Be the first to comment

Tool Driven Restructuring of Large Codebases

  1. 1. TOOL-DRIVEN RESTRUCTURING OF LARGE CODEBASES ganesh samarthyam (ganesh@codeops.tech)
  2. 2. FOCUS ON REPAYING ARCHITECTURE DEBT HERE … Code debt Static analysis tool violations Coding style violations Design debt Design smells Violations of design rules Test debt Lack of tests Inadequate test coverage Architecture debt Lacking structure (esp. tangles) Violation of architecture constraints our focus is mainly here…
  3. 3. UNDERLYING PRINCIPLES • First: Prefer non-invasive architecture restructuring instead of invasive code changes • Second: “Automation” – rely on combination of tools for automated improvement • Third: “Assure” – prefactor to practice and show its safe and gives great benefits
  4. 4. LARGE (LEGACY) CODE ➤ Positives: ➤ Working code ➤ Tried and tested ➤ Stable & known ➤ … ➤ Negatives: ➤ “Touch here breaks there” (lack of visibility & deteriorating structure) ➤ No ONE knows how it works (but we do know it works!) ➤ Limited resources (unclear investment benefits) ➤ …
  5. 5. CORE ISSUE: DIFFICULT TO UNDERSTAND & LACKS STRUCTURE ➤ Lack of shared understanding of the whole system ➤ Original developers have long left the team ➤ New developers take a long time to become productive ➤ Structure has deteriorated over time (existing structure works against development) ➤ Accidental changes to the structure (on need basis) than intentional changes that preserves structure ➤ Changing code is “perceived risky” - hard to predict impact ➤ As developers we don’t know how to fix the code or add features; when we know, can’t be sure if it won’t break anything (lack of automated unit tests adds to the problem)
  6. 6. Source: Consortium of IT Software Quality (CISQ), Bill Curtis, Architecturally Complex Defects, 2012
  7. 7. WHICH ONE WOULD YOU LIKE TO WORK ON? V 0.7.2 V 1.3.5 https://sourceforge.net/projects/findbugs/files/findbugs/
  8. 8. STARTS WITH AN UNDERSTANDABLE INTENSIONAL STRUCTURE V 0.7.2
  9. 9. GETS SLIGHTLY COMPLICATED V 0.8.6
  10. 10. FIRST TANGLE APPEARS V 0.8.7
  11. 11. AND THE TANGLE SPREADS V 0.8.8
  12. 12. ITS GETTING TOUGHER TO DEAL WITH V 1.0.0
  13. 13. IT IS GETTING WORSER V 1.3.0
  14. 14. ITS A MESS NOW! V 1.3.5
  15. 15. HOW DO WE DEAL WITH THIS SITUATION?
  16. 16. DOES MODULARITY MATTER? Tangles in JDK
  17. 17. “MODULARIZED” SOLUTION (JDK 9)
  18. 18. MODULARITY IS THE KEY: REMOVING TANGLES AND FAT ➤ Tangles - entities are directly or indirectly depending on each other (classes, packages, etc) ➤ Fat - excessive complexity of an entity (methods, classes, packages etc)
  19. 19. ARE YOU IN THIS SITUATION? => DO THIS!
  20. 20. ARE YOU IN THIS SITUATION? => DO THIS!
  21. 21. UNDERSTANDING PHYSICAL/CLASS TANGLES Step 1: Separate the interface & implementation Step 2: Depend on the interfaces (and not on the implementations) (Reflection: Needs code-level changes)
  22. 22. UNDERSTANDING PACKAGE-LEVEL TANGLES Step 1: Create a separate package for the interface(s) Step 2: Separate the interface & implementation (Reflection: Needs package-moves - no code level changes)
  23. 23. HOW TO EXTRACT A MODULE FROM A TANGLE?
  24. 24. HOW TO EXTRACT A MODULE FROM A TANGLE?
  25. 25. SEPARATE INTERFACE & IMPL - FOR ‘CLASSREALM’ (WITHOUT CYCLE) Step: Separate the interfaces in “maven-classrealm” from the implementation (concrete classes) into to a separate module (“maven-classrealm-api”) (Reflection: NO CODE CHANGE(S) NEEDED!!)
  26. 26. STORY OF HOW BUILD TOOL GOT EXTRACTED Smell Detector tool (UI & implementation logic) Request came in for build-time checking tool Smell Detector Tool (UI) Smell Detector Build Tool (headless) Smell Detector Implementation Logic (DLL) Interface
  27. 27. WHAT IS “PRESTRUCTURING”?
  28. 28. WHAT IS PRESTRUCTURING? ➤ Jack is “chained” – aka “tangled” ;-) ➤ Rose is scared to use axe because too risky and lacks experience – aka “fear of breaking the working code” - hence practices first! ➤ “Rehearse restructuring the codebase without actually making the code changes!” ➤ Safely try out the specific structural changes ➤ Evaluate benefits & trade-offs ➤ Undo if needed ➤ Discuss and brainstorm by “trying out things” ➤ Plan the actual restructuring ➤ To the specific / exact steps and details
  29. 29. REFACTORING VS. RESTRUCTURING Code Refactoring Modular Restructuring It is improving the underlying design of existing code through code modifications to improve quality of the software, esp. maintainability. Improving the structure of the code-base with little or no modifications to the existing code to improve software modularity. Target of change: Code itself. Refactoring is done directly to the code (physical level). Target of change: Code base. Changes are done largely at higher levels of abstractions (logical level). Extent of code changes: A lot of invasive code editing. No code editing in most scenarios; sometimes minimal code editing is needed for specific structural issues. What to refactor: Smells (A code smell is a surface indication that usually corresponds to a deeper problem in the system). What to restructure: Structural issues (dependencies, levelization, placement, size, grouping, etc.). Small worlds of a few classes at a time; what you see in the IDE (examples: data classes, large methods, long parameter lists) Whole codebase; what you don’t see in the IDE (examples: tangles and cycles, orphan packages, misplaced class in a package)
  30. 30. BENEFITS - MODULAR RESTRUCTURING Developer Productivity (quickening mental models when working with code) The time to understand the structure of a large codebase (especially for teams who have been transferred responsibility of an existing product) is drastically reduced The cost of transfer of maintenance is lesser as KT time reduced as much as 75% Developer Productivity (modularization results in reduced change impact) The time to make a change (new enhancement/bug fix) is drastically reduced as the time to do impact analysis is drastically reduced The bug density is lower as developers can better modularize and make changes confidently Developer Productivity (improved build times) Modularization results in reduced time for compilation & more modular components as part of a modular build system Faster build times (even upto 90%) resulting in quicker turnaround time for developers for changes + quicker unit testing
  31. 31. CATALOG OF RESTRUCTURINGS
  32. 32. “FATTY PACKAGE” – TOO MANY CLASSES IN A PACKAGE Potential cause Missing packaging of classes Restructuring Step 1: Auto-group and levelized Step 2: Identify technical or functional groups Step 3: Wrap them into sub-packages
  33. 33. “MISSING PACKAGING” When there are few / no tangles in a package and there are many classes, it indicates that subpackaging is missing Pulling from bottom or top as packaging exposes new opportunities for subpackaging So perform restructuring iteratively (not in one go)
  34. 34. “MISPLACED CLASS(ES)” Restructuring Move it to the suitable container 
  35. 35. “TROUBLEMAKER CLASS” Structural issue: A class should logically belong to the current package but causes cyclic dependency if left as it is! Potential cause: Implementation code (perhaps inadvertently) results in a cycle;  Restructuring:   Option 1: Refactor the implementation code (not recommended in this case) Option 2: Move it outside the package (simple and effective solution in this case)
  36. 36. “MISPLACED-TROUBLEMAKER CLASS!” Structural issue: A class should logically belong to the suitable package but causes cyclic dependency if moved to that package Potential cause: Implementation code (perhaps inadvertently) results in a cycle Restructuring:   Option 1: Refactor the implementation code (not recommended in this case) Option 2: Move it outside the package (simple and effective solution in this case)
  37. 37. “ORPHANS” Structural issue Unreferenced entities in the code base Potential causes Historic nature – used earlier but not anymore Left-overs – created but never used Caution It may be actually used but does not have dependencies visible at compiled- code level Through features like reflection or as APIs Or using mechanisms external to the language Restructuring  Step 1: Determine that they are actually unused Step 2: Remove them Your chance to commit “pre-mediated murders of innocent orphans” and become a “killer coder” ;-)
  38. 38. “KNOTTY PACKAGES” Structural issue Packages are too much tangled  Potential cause Haphazard organization without thinking Degradation over time  Restructuring – “unwrap and regroup” Step 1: Unwrap packages Step 2: Regroup classes (“Auto- regroup”) Step 3: Wrap into proper packages (“Realize”) Tool hint: Try “auto-levelization” magic!
  39. 39. “KNOTTY CLASSES” Structural issue Classes within a package are too much tangled  Potential cause Lack of grouping/sub-packaging Restructuring  Step 1: Autogroup classes Step 2: Wrap relevant groups into sub-packages (“Realize”)
  40. 40. “UNTIDY PACKAGE” Structural issue Both fat and tangles in the package Potential cause Haphazard organization (putting things in a package without thinking about structure) Restructuring  Step 1: Unwrap everything the package to class level Step 2: Identify groups / cohesive clusters Step 3: Repackage with new packages
  41. 41. MAKING “PRAGMATIC CHOICES” IN RESTRUCTURING ThreadLocalUtility causes tangles “Used by”: “async” and “webapp” “Uses”: “async”, “context”, and “webapp”  Going by the names of the entities “thread-local-utility” best belongs to “async” but move it to “webapp” But if we move it there, it will cause tangle between “async” and “webapp” Perfectionist approach will move it to “async” and also do “code refactoring” to get rid of the tangle – why should ThreadLocalUtility depend on “webapp” or “context”? Arguably pragmatic approach: Move it to “webapp” because it will resolve tangle though it doesn’t belong there
  42. 42. FUNDAMENTAL MODULARISATION PRICIPLES Modularization Levelization / Layering Grouping Chunking Encapsulation / hiding Ordering
  43. 43. “Prefer “a move” (code restructuring) instead of “invasive code change” (code refactoring) A BIG IDEA!
  44. 44. TOOLS FOR REFACTORING & RESTRUCTURING ➤ CodeScene ➤ Designite ➤ JArchitect / NDepend / CppDepend ➤ Lattix ➤ ReSharper ➤ Sotograph / SotoArc ➤ and more …
  45. 45. BIG IDEAS AND TAKEAWAYS ➤ It is possible to modularize WITHOUT changing any code! ➤ Prefer making non-invasive architecture restructuring (e.g., class moves to remove a package tangle) instead of making invasive code changes (e.g., change the code inside a class to remove the offending dependency) ➤ Tools are your friends - befriend them :-)
  46. 46. APPLYING PRINCIPLES
  47. 47. CREDITS ➤ For the FindBugs example: Structure101 team ➤ For applying principles image: our book “Refactoring for Software Design Smells” ➤ Module system in JDK 9 - blogs.oracle.com ➤ Modular restructuring benefits: Manoj Ganapathi (CodeOps) ➤ For the “maven class-realm” module extraction case study: Mike Swainston-Rainford ➤ Source for the quotation: Consortium of IT Software Quality (CISQ), Bill Curtis, Architecturally Complex Defects, 2012
  48. 48. IMAGE CREDITS ➤ https://pixabay.com/illustrations/lion-image-editing-graphic- program-66898/ ➤ https://pixabay.com/photos/roofs-houses-historic-center- italy-919460/
  49. 49. www.codeops.tech www.konfhub.com www.designsmells.com ganesh@codeops.tech bit.ly/ganeshsg @GSamarthyam

×