Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Devnology LectureSoftware ReengineeringMartin PinzgerSoftware Engineering Research GroupDelft University of Technology
Greenfield software development                                 2
Non-greenfield software development            ?                                     3
How often did you ...... encounter greenfield and non-greenfield softwareengineering?                                       ...
Why non-greenfield engineering?Because existing software, often called legacy software, isvaluable  Often business-critical...
Why do we (often) start from a mess?                                       6
Lehman’s Laws of software evolutionContinuing change  A program that is used in a real-world environment must change, or  ...
Evolution of Mozilla source code                                   8
Lehman’s Laws in practiceExisting software Is often modified in an ad-hoc manner (quickfixes)  Lack of time, resources, mone...
Typical result of such practices                                   10
Implications of the resultsSoftware maintenance costs continuously increaseBetween 50% and 75% of global software developm...
What is your decision?            According to Lehman: “there will always be changes”                                 hack...
Lecture OutlineIntroductionReengineering ProcessModule Capture/Reverse EngineeringProblem DetectionSummary and Conclusions...
Let’s reengineerDefinition:“Reengineering is the examination and alteration of a subjectsystem to reconstitute it in a new ...
Reengineering Life-Cycle               (1) requirement    New                    analysis      Requirements (3) problem  d...
Goals of reengineering                         Testability                         Understandability                      ...
Goals of reengineering (2)Unbundling  Split a monolithic system into parts that can be separately marketedPerformance  “Fi...
In this course, you will learnBest practices to analyze and understand software systems(i.e., reverse engineering)Heuristi...
Model CaptureReverse EngineeringSetting DirectionFirst ContactInitial UnderstandingDetailed Model Capture
Setting direction patterns             Set direction                                                            Where to s...
Pattern: Most Valuable FirstProblem: Which problems should you address first?                                              ...
Most valuable first (2)Solution: Work on aspects that are most valuable to yourcustomer  Maximize commitment  Deliver resul...
Most valuable first (3)                         23
Most valuable first (4)How do you tell what is valuable?  Identify your customer  Understand the customer’s business model ...
Most valuable first (5)Planning Game                         25
Model CaptureReverse EngineeringSetting DirectionFirst ContactInitial UnderstandingDetailed Model Capture
What is Reverse Engineering and why?Reverse Engineering is the process of analyzing a subject system  to identify the syst...
First contact patterns                                                                            feasibility assessment  ...
Pattern: Read all the code in one hourProblem: Yes, but… the system is so big! Where to start?                            ...
Read all the code in one hour (2)Solution: Read the code in one hourFocus on:  Functional tests and unit tests  Abstract c...
In Java programs focus onpublic abstract class Example {    public interface IExample {...                                ...
First project planProject scope (1/2 page)  Description, context, goals, verification criteriaOpportunities  Identify facto...
Model CaptureReverse EngineeringSetting DirectionFirst ContactInitial UnderstandingDetailed Model Capture
Initial understanding patterns                            Top down                                                 Recover...
Study the exceptional entitiesProblem: How can you quickly identify design problems?Solution: Measure software entities an...
Example: Exceptional entities                                Use simple                                Use simple         ...
Model CaptureReverse EngineeringSetting DirectionFirst ContactInitial UnderstandingDetailed Model Capture
Detailed model capture patterns       Tie Code and Questions                                Expose the design & make sure ...
Refactor to understandProblem: How do you decipher cryptic code?Solution: Refactor it till it makes sense  Goal (for now) ...
Refactor to understand (cont.)Guidelines  Rename attributes to convey roles  Rename methods and classes to reveal intent  ...
Learn from the pastProblem: How did the system get the way it is? Which parts arestable and which aren’t?Solution: Compare...
Examples: Unstable designPulsar: Repeated Modifications make it grow and shrink.System Hotspot: Every System Version requi...
Reverse engineering tools/prototypesX-Ray  http://xray.inf.usi.ch/xray.phpDA4Java  http://swerl.tudelft.nl/bin/view/Martin...
X-Ray        44
DA4Java          45
CodeCity           46
Summary Model CaptureSetting direction patterns to  Set the goals  Find the Go/No-Go decision  Increase commitment of clie...
Problem DetectionIn the Source CodeIn the Evolution
Design problemsThe most common design problems result from code that isUnclear & complicated             Duplicated (code ...
Code Smells (if it stinks, change it)A code smell is a hint that something has gone wrongsomewhere in your code.Duplicated...
How to detect?Measure and visualize quality aspects of the currentimplementation of a system  Source code metrics and stru...
Polymetric ViewsA combination of metrics and softwarevisualization                                         Entity  Visuali...
Smell 1: Long MethodThe longer a method is, the more difficult it is to understandit.When is a method too long?  Heuristic:...
Method Length Distribution                       Metrics:                       Boxes: Methods                       Width...
Smell 2: Switch StatementProblem is similar to code duplication  Switch statement is scattered in different placesHow to d...
Method ComplexityMetrics:Boxes: MethodsPosition-X: LOCPosition-Y: MCCSort: -                    56
More info on Detection StrategiesObject-Oriented Metrics in PracticeMichele Lanza and Radu Marinescu, Springer 2006http://...
Tool for Smell DetectioninCode http://www.intooitus.com/inCode.htmljDeodorant http://java.uom.gr/~jdeodorant/             ...
Problem DetectionIn the Source CodeIn the Evolution
Understanding EvolutionChanges can point to design problems  “Evolutionary Smells”But  Overwhelming complexity  How can we...
Visualizing Class EvolutionVisualize classes as rectangles using forwidth and height the following metrics:      Foo  NOM ...
The Evolution Matrix                                  Removed Classes   Last VersionFirst Version     Added     Classes   ...
Evolution Patterns & SmellsDay-fly (Dead Code)PersistentPulsar (Change Prone Entity)SupernovaWhite Dwarf (Dead Code)Red Gia...
Persistent / Dayfly                            Dayflies: Exists                            during only one or              ...
Pulsar / Supernova Pulsar: Repeated Modifications make it grow and shrink. System Hotspot: Every System Version requires c...
White Dwarf / Red Giant / IdleWhite Dwarf: Lost the functionality it had and now trundles along withoutreal meaning. Possi...
Summary Problem DetectionDesign Problems  Result from duplicated, unclear, complicated source code  -> Code SmellsDetectio...
ConclusionsObject-Oriented Re-engineering Patterns  Set of best practices to re-engineering software systemsModule Capture...
Reading material                         Object-Oriented Reengineering Patterns                         Serge Demeyer, Ste...
Additional reading                         Agile Software Development: Principles Patterns, and Practices                 ...
Upcoming SlideShare
Loading in …5
×

Devnology back toschool software reengineering

1,725 views

Published on

Lecture for Devnology's Back to School program about software re-engineering using visualization techniques

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Devnology back toschool software reengineering

  1. 1. Devnology LectureSoftware ReengineeringMartin PinzgerSoftware Engineering Research GroupDelft University of Technology
  2. 2. Greenfield software development 2
  3. 3. Non-greenfield software development ? 3
  4. 4. How often did you ...... encounter greenfield and non-greenfield softwareengineering? 4
  5. 5. Why non-greenfield engineering?Because existing software, often called legacy software, isvaluable Often business-critical A huge amount of money has already been invested in it Has been tested and runs Does (mainly) what it should doWould you replace such a system? 5
  6. 6. Why do we (often) start from a mess? 6
  7. 7. Lehman’s Laws of software evolutionContinuing change A program that is used in a real-world environment must change, or become progressively less useful in that environment.Increasing complexity As a program evolves, it becomes more complex, and extra resources are needed to preserve and simplify its structure.For more information read Lehman and Belady, 1985 7
  8. 8. Evolution of Mozilla source code 8
  9. 9. Lehman’s Laws in practiceExisting software Is often modified in an ad-hoc manner (quickfixes) Lack of time, resources, money, etc.Initial good design is not maintained Spaghetti code, copy/paste programming, dependencies are introduced, no tests, etc.Documentation is not updated (if there is one) Architecture and design documentsOriginal developers leave and with them their knowledge 9
  10. 10. Typical result of such practices 10
  11. 11. Implications of the resultsSoftware maintenance costs continuously increaseBetween 50% and 75% of global software development costsare spent on maintenance!Up to 60% of a maintenance effort is spent on understandingthe existing software 11
  12. 12. What is your decision? According to Lehman: “there will always be changes” hack it?* duplicated code * first reengineer* complex conditionals * then implement changes* abusive inheritance* large classes/methodsTake a loan on your software Investment for the futurepay back via reengineering paid back during maintenance 12
  13. 13. Lecture OutlineIntroductionReengineering ProcessModule Capture/Reverse EngineeringProblem DetectionSummary and Conclusions 13
  14. 14. Let’s reengineerDefinition:“Reengineering is the examination and alteration of a subjectsystem to reconstitute it in a new form and the subsequentimplementation of the new form.”[Demeyer, Ducasse, Nierstrasz]http://scg.unibe.ch/download/oorp/ 14
  15. 15. Reengineering Life-Cycle (1) requirement New analysis Requirements (3) problem detection (4) problem resolution Designs(2) model capture Code 15
  16. 16. Goals of reengineering Testability Understandability Modifiability Extensibility Maintainability … 16
  17. 17. Goals of reengineering (2)Unbundling Split a monolithic system into parts that can be separately marketedPerformance “First do it, then do it right, then do it fast”Design extraction To improve maintainability, portability, etc.Exploitation of New Technology I.e., new language features, standards, libraries, etc. 17
  18. 18. In this course, you will learnBest practices to analyze and understand software systems(i.e., reverse engineering)Heuristics and tools to detect shortcomings in the design andimplementation of software systems 18
  19. 19. Model CaptureReverse EngineeringSetting DirectionFirst ContactInitial UnderstandingDetailed Model Capture
  20. 20. Setting direction patterns Set direction Where to start? Agree on Maxims Maintain Coordinate direction direction Most Valuable First Appoint a Speak to the Navigator Round Table What to do? What not to do? Fix Problems, If It Aint Broke Not Symptoms Dont Fix ItPrinciples & guidelines forsoftware project management are How to do it?especially relevant for reengineeringprojects Keep it Simple 20
  21. 21. Pattern: Most Valuable FirstProblem: Which problems should you address first? 21
  22. 22. Most valuable first (2)Solution: Work on aspects that are most valuable to yourcustomer Maximize commitment Deliver results early Build confidence 22
  23. 23. Most valuable first (3) 23
  24. 24. Most valuable first (4)How do you tell what is valuable? Identify your customer Understand the customer’s business model Determine measurable goals Consult change logs for high activity Play the Planning Game Fix Problems, not Symptoms 24
  25. 25. Most valuable first (5)Planning Game 25
  26. 26. Model CaptureReverse EngineeringSetting DirectionFirst ContactInitial UnderstandingDetailed Model Capture
  27. 27. What is Reverse Engineering and why?Reverse Engineering is the process of analyzing a subject system to identify the system’s components and their interrelationships and create representations of the system in another form or at a higher level of abstraction [Chikofsky & Cross, ’90]Motivation Understanding other people’s code, the design and architecture in order to maintain and evolve a software system 27
  28. 28. First contact patterns feasibility assessment System experts (one week time) Talk with Talk with developers end users Chat with the Interview Talk about it Maintainers during Demo Software System Verify what you hear Read about Read it Compile it it Read All the Code Skim the Do a Mock in One Hour Documentation Installation 28
  29. 29. Pattern: Read all the code in one hourProblem: Yes, but… the system is so big! Where to start? 29
  30. 30. Read all the code in one hour (2)Solution: Read the code in one hourFocus on: Functional tests and unit tests Abstract classes and methods and classes high in the hierarchy Surprisingly large structures Comments Check classes with high fan-out Study the build process 30
  31. 31. In Java programs focus onpublic abstract class Example { public interface IExample {... ...} }/** public class Test { * Block comment ... */ @Testpublic class Example { public void testExample() { public void foo() { ... int x = 1; } for (int x=1; i<100; i++) { } // do something comment } }} 31
  32. 32. First project planProject scope (1/2 page) Description, context, goals, verification criteriaOpportunities Identify factors to achieve project goals Skilled maintainers, readable source-code, documentation, etc.Risks Identify risks that may cause problems Absent test-suites, missing libraries, etc. Record likelihood & impact for each riskGo/no-go decision, activities (fish-eye view) 32
  33. 33. Model CaptureReverse EngineeringSetting DirectionFirst ContactInitial UnderstandingDetailed Model Capture
  34. 34. Initial understanding patterns Top down Recover design Speculate about Design ITERATION understand ⇒ higher-level model Analyze the Study the Persistent Data Exceptional Entities Recover Identify database problems Bottom up 34
  35. 35. Study the exceptional entitiesProblem: How can you quickly identify design problems?Solution: Measure software entities and study the anomalous ones Visualize metrics to get an overview Use simple metrics Lines of code Number of methods ... 35
  36. 36. Example: Exceptional entities Use simple Use simple metrics and metrics and layout layout algorithms algorithms. height colour (x,y) width Visualize up to 5 metrics per node 36
  37. 37. Model CaptureReverse EngineeringSetting DirectionFirst ContactInitial UnderstandingDetailed Model Capture
  38. 38. Detailed model capture patterns Tie Code and Questions Expose the design & make sure it stays exposed Expose design Keep track of your understanding Refactor to Understand Expose collaborations Step through the Execution Expose contracts Write Tests to Understand Look for the Contracts• Use Your Tools• Look for Key Methods Expose evolution• Look for Constructor Calls• Look for Template/Hook Methods Learn from the Past• Look for Super Calls 38
  39. 39. Refactor to understandProblem: How do you decipher cryptic code?Solution: Refactor it till it makes sense Goal (for now) is to understand, not to reengineerHints Work with a copy of the code Refactoring requires an adequate test base If this is missing, “Write Tests to Understand” 39
  40. 40. Refactor to understand (cont.)Guidelines Rename attributes to convey roles Rename methods and classes to reveal intent Remove duplicated code Replace condition branches by methods 40
  41. 41. Learn from the pastProblem: How did the system get the way it is? Which parts arestable and which aren’t?Solution: Compare versions to discover where code was removed Removed functionality is a sign of design evolution Use or develop appropriate tools Look for signs of: Unstable design — repeated growth and refactoring Mature design — growth, refactoring, and stability 41
  42. 42. Examples: Unstable designPulsar: Repeated Modifications make it grow and shrink.System Hotspot: Every System Version requires changes. 42
  43. 43. Reverse engineering tools/prototypesX-Ray http://xray.inf.usi.ch/xray.phpDA4Java http://swerl.tudelft.nl/bin/view/MartinPinzger/MartinPinzgerDA4JavaCode City http://www.inf.usi.ch/phd/wettel/codecity.html 43
  44. 44. X-Ray 44
  45. 45. DA4Java 45
  46. 46. CodeCity 46
  47. 47. Summary Model CaptureSetting direction patterns to Set the goals Find the Go/No-Go decision Increase commitment of clients and developersFirst contact patterns to Obtain an overview and grasp the main issues Assess the feasibility of the projectInitial Understanding & Detailed Model Capture Plan the work … and work the plan Frequent and short iterations 47
  48. 48. Problem DetectionIn the Source CodeIn the Evolution
  49. 49. Design problemsThe most common design problems result from code that isUnclear & complicated Duplicated (code clones) 49
  50. 50. Code Smells (if it stinks, change it)A code smell is a hint that something has gone wrongsomewhere in your code.Duplicated CodeLong MethodLarge ClassLong Parameter ListDivergent ChangeShotgun SurgeryFeature Envy... 50
  51. 51. How to detect?Measure and visualize quality aspects of the currentimplementation of a system Source code metrics and structuresMeasure and visualize quality aspects of the evolution of asystem Evolution metrics and structuresUse Polymetric Views 51
  52. 52. Polymetric ViewsA combination of metrics and softwarevisualization Entity Visualize software using colored rectangles for the entities and edges for the relationships Relationship Render up to five metrics on one node: Size (1+2) Color (3) Position (4+5) X Coordinate Y Coordinate Color tone Height Width 7 52
  53. 53. Smell 1: Long MethodThe longer a method is, the more difficult it is to understandit.When is a method too long? Heuristic: > 10 LOCs (?)How to detect? Visualize LOC metric values of methods “Method Length Distribution View” 53
  54. 54. Method Length Distribution Metrics: Boxes: Methods Width: LOC Position-Y: LOC Sort: LOC 54
  55. 55. Smell 2: Switch StatementProblem is similar to code duplication Switch statement is scattered in different placesHow to detect? Visualize McCabe Cyclomatic Complexity metric to detect complex methods “Method Complexity Distribution View” 55
  56. 56. Method ComplexityMetrics:Boxes: MethodsPosition-X: LOCPosition-Y: MCCSort: - 56
  57. 57. More info on Detection StrategiesObject-Oriented Metrics in PracticeMichele Lanza and Radu Marinescu, Springer 2006http://www.springer.com/computer/swe/book/978-3-540-24429-5 57
  58. 58. Tool for Smell DetectioninCode http://www.intooitus.com/inCode.htmljDeodorant http://java.uom.gr/~jdeodorant/ 58
  59. 59. Problem DetectionIn the Source CodeIn the Evolution
  60. 60. Understanding EvolutionChanges can point to design problems “Evolutionary Smells”But Overwhelming complexity How can we detect and understand changes?Solutions The Evolution Matrix The Kiviat Graphs 60
  61. 61. Visualizing Class EvolutionVisualize classes as rectangles using forwidth and height the following metrics: Foo NOM (number of methods) NOA (number of attributes) BarThe Classes can be categorized accordingto their “personal evolution” and to their“system evolution”-> Evolution Patterns 61
  62. 62. The Evolution Matrix Removed Classes Last VersionFirst Version Added Classes Major Leap Growth Stabilisation TIME (Versions) 62
  63. 63. Evolution Patterns & SmellsDay-fly (Dead Code)PersistentPulsar (Change Prone Entity)SupernovaWhite Dwarf (Dead Code)Red Giant (Large/God Class)Idle (Dead Code) 63
  64. 64. Persistent / Dayfly Dayflies: Exists during only one or two versions. Perhaps Persistent: Has the same an idea which was lifespan as the whole tried out and then system. Part of the dropped. original design. Perhaps holy dead code which no one dares to remove. 64
  65. 65. Pulsar / Supernova Pulsar: Repeated Modifications make it grow and shrink. System Hotspot: Every System Version requires changes. Supernova: Sudden increase in size. Possible Reasons: • Massive shift of functionality towards a class. • Data holder class for which it is easy to grow. • Sleeper: Developers knew exactly what to fill in. 65
  66. 66. White Dwarf / Red Giant / IdleWhite Dwarf: Lost the functionality it had and now trundles along withoutreal meaning. Possibly dead code -> Lazy Class.Red Giant: A permanent god (large) class which is always very large.Idle: Keeps size over several versions. Possibly dead code,possibly good code. 66
  67. 67. Summary Problem DetectionDesign Problems Result from duplicated, unclear, complicated source code -> Code SmellsDetection heuristics and Polymetric Views to detect code andevolution smells 67
  68. 68. ConclusionsObject-Oriented Re-engineering Patterns Set of best practices to re-engineering software systemsModule Capture and Reverse Engineering Understand the design and implementation of software systemsProblem Detection Heuristics to detect Bad Smells in the source code and evolution of software systemsNext Step Add tests and refactor detected problems 68
  69. 69. Reading material Object-Oriented Reengineering Patterns Serge Demeyer, Stephane Ducasse, and Oscar Nierstrasz free copy from: http://scg.unibe.ch/download/oorp/ Working Effectively with Legacy Code Michael Feathers, Prentice Hall, 1 edition, 2004Refactoring to PatternsJoshua Kerievsky, Addison-Wesley Professional, 2004 69
  70. 70. Additional reading Agile Software Development: Principles Patterns, and Practices Robert C. Martin, Prentice Hall Object-Oriented Design Heuristics Arthur J. Riel, Prentice Hall, 1 edition, 1996Refactoring: Improving the Design of Existing CodeMartin Fowler, Addison-Wesley Professional, 1999 70

×