Your SlideShare is downloading. ×
Devnology back toschool software reengineering
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Devnology back toschool software reengineering


Published on

Lecture for Devnology's Back to School program about software re-engineering using visualization techniques

Lecture for Devnology's Back to School program about software re-engineering using visualization techniques

Published in: Technology

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Devnology LectureSoftware ReengineeringMartin PinzgerSoftware Engineering Research GroupDelft University of Technology
  • 2. Greenfield software development 2
  • 3. Non-greenfield software development ? 3
  • 4. How often did you ...... encounter greenfield and non-greenfield softwareengineering? 4
  • 5. Why non-greenfield engineering?Because existing software, often called legacy software, isvaluable Often business-critical A huge amount of money has already been invested in it Has been tested and runs Does (mainly) what it should doWould you replace such a system? 5
  • 6. Why do we (often) start from a mess? 6
  • 7. Lehman’s Laws of software evolutionContinuing change A program that is used in a real-world environment must change, or become progressively less useful in that environment.Increasing complexity As a program evolves, it becomes more complex, and extra resources are needed to preserve and simplify its structure.For more information read Lehman and Belady, 1985 7
  • 8. Evolution of Mozilla source code 8
  • 9. Lehman’s Laws in practiceExisting software Is often modified in an ad-hoc manner (quickfixes) Lack of time, resources, money, etc.Initial good design is not maintained Spaghetti code, copy/paste programming, dependencies are introduced, no tests, etc.Documentation is not updated (if there is one) Architecture and design documentsOriginal developers leave and with them their knowledge 9
  • 10. Typical result of such practices 10
  • 11. Implications of the resultsSoftware maintenance costs continuously increaseBetween 50% and 75% of global software development costsare spent on maintenance!Up to 60% of a maintenance effort is spent on understandingthe existing software 11
  • 12. What is your decision? According to Lehman: “there will always be changes” hack it?* duplicated code * first reengineer* complex conditionals * then implement changes* abusive inheritance* large classes/methodsTake a loan on your software Investment for the futurepay back via reengineering paid back during maintenance 12
  • 13. Lecture OutlineIntroductionReengineering ProcessModule Capture/Reverse EngineeringProblem DetectionSummary and Conclusions 13
  • 14. Let’s reengineerDefinition:“Reengineering is the examination and alteration of a subjectsystem to reconstitute it in a new form and the subsequentimplementation of the new form.”[Demeyer, Ducasse, Nierstrasz] 14
  • 15. Reengineering Life-Cycle (1) requirement New analysis Requirements (3) problem detection (4) problem resolution Designs(2) model capture Code 15
  • 16. Goals of reengineering Testability Understandability Modifiability Extensibility Maintainability … 16
  • 17. Goals of reengineering (2)Unbundling Split a monolithic system into parts that can be separately marketedPerformance “First do it, then do it right, then do it fast”Design extraction To improve maintainability, portability, etc.Exploitation of New Technology I.e., new language features, standards, libraries, etc. 17
  • 18. In this course, you will learnBest practices to analyze and understand software systems(i.e., reverse engineering)Heuristics and tools to detect shortcomings in the design andimplementation of software systems 18
  • 19. Model CaptureReverse EngineeringSetting DirectionFirst ContactInitial UnderstandingDetailed Model Capture
  • 20. Setting direction patterns Set direction Where to start? Agree on Maxims Maintain Coordinate direction direction Most Valuable First Appoint a Speak to the Navigator Round Table What to do? What not to do? Fix Problems, If It Aint Broke Not Symptoms Dont Fix ItPrinciples & guidelines forsoftware project management are How to do it?especially relevant for reengineeringprojects Keep it Simple 20
  • 21. Pattern: Most Valuable FirstProblem: Which problems should you address first? 21
  • 22. Most valuable first (2)Solution: Work on aspects that are most valuable to yourcustomer Maximize commitment Deliver results early Build confidence 22
  • 23. Most valuable first (3) 23
  • 24. Most valuable first (4)How do you tell what is valuable? Identify your customer Understand the customer’s business model Determine measurable goals Consult change logs for high activity Play the Planning Game Fix Problems, not Symptoms 24
  • 25. Most valuable first (5)Planning Game 25
  • 26. Model CaptureReverse EngineeringSetting DirectionFirst ContactInitial UnderstandingDetailed Model Capture
  • 27. What is Reverse Engineering and why?Reverse Engineering is the process of analyzing a subject system to identify the system’s components and their interrelationships and create representations of the system in another form or at a higher level of abstraction [Chikofsky & Cross, ’90]Motivation Understanding other people’s code, the design and architecture in order to maintain and evolve a software system 27
  • 28. First contact patterns feasibility assessment System experts (one week time) Talk with Talk with developers end users Chat with the Interview Talk about it Maintainers during Demo Software System Verify what you hear Read about Read it Compile it it Read All the Code Skim the Do a Mock in One Hour Documentation Installation 28
  • 29. Pattern: Read all the code in one hourProblem: Yes, but… the system is so big! Where to start? 29
  • 30. Read all the code in one hour (2)Solution: Read the code in one hourFocus on: Functional tests and unit tests Abstract classes and methods and classes high in the hierarchy Surprisingly large structures Comments Check classes with high fan-out Study the build process 30
  • 31. In Java programs focus onpublic abstract class Example { public interface IExample {... ...} }/** public class Test { * Block comment ... */ @Testpublic class Example { public void testExample() { public void foo() { ... int x = 1; } for (int x=1; i<100; i++) { } // do something comment } }} 31
  • 32. First project planProject scope (1/2 page) Description, context, goals, verification criteriaOpportunities Identify factors to achieve project goals Skilled maintainers, readable source-code, documentation, etc.Risks Identify risks that may cause problems Absent test-suites, missing libraries, etc. Record likelihood & impact for each riskGo/no-go decision, activities (fish-eye view) 32
  • 33. Model CaptureReverse EngineeringSetting DirectionFirst ContactInitial UnderstandingDetailed Model Capture
  • 34. Initial understanding patterns Top down Recover design Speculate about Design ITERATION understand ⇒ higher-level model Analyze the Study the Persistent Data Exceptional Entities Recover Identify database problems Bottom up 34
  • 35. Study the exceptional entitiesProblem: How can you quickly identify design problems?Solution: Measure software entities and study the anomalous ones Visualize metrics to get an overview Use simple metrics Lines of code Number of methods ... 35
  • 36. Example: Exceptional entities Use simple Use simple metrics and metrics and layout layout algorithms algorithms. height colour (x,y) width Visualize up to 5 metrics per node 36
  • 37. Model CaptureReverse EngineeringSetting DirectionFirst ContactInitial UnderstandingDetailed Model Capture
  • 38. Detailed model capture patterns Tie Code and Questions Expose the design & make sure it stays exposed Expose design Keep track of your understanding Refactor to Understand Expose collaborations Step through the Execution Expose contracts Write Tests to Understand Look for the Contracts• Use Your Tools• Look for Key Methods Expose evolution• Look for Constructor Calls• Look for Template/Hook Methods Learn from the Past• Look for Super Calls 38
  • 39. Refactor to understandProblem: How do you decipher cryptic code?Solution: Refactor it till it makes sense Goal (for now) is to understand, not to reengineerHints Work with a copy of the code Refactoring requires an adequate test base If this is missing, “Write Tests to Understand” 39
  • 40. Refactor to understand (cont.)Guidelines Rename attributes to convey roles Rename methods and classes to reveal intent Remove duplicated code Replace condition branches by methods 40
  • 41. Learn from the pastProblem: How did the system get the way it is? Which parts arestable and which aren’t?Solution: Compare versions to discover where code was removed Removed functionality is a sign of design evolution Use or develop appropriate tools Look for signs of: Unstable design — repeated growth and refactoring Mature design — growth, refactoring, and stability 41
  • 42. Examples: Unstable designPulsar: Repeated Modifications make it grow and shrink.System Hotspot: Every System Version requires changes. 42
  • 43. Reverse engineering tools/prototypesX-Ray City 43
  • 44. X-Ray 44
  • 45. DA4Java 45
  • 46. CodeCity 46
  • 47. Summary Model CaptureSetting direction patterns to Set the goals Find the Go/No-Go decision Increase commitment of clients and developersFirst contact patterns to Obtain an overview and grasp the main issues Assess the feasibility of the projectInitial Understanding & Detailed Model Capture Plan the work … and work the plan Frequent and short iterations 47
  • 48. Problem DetectionIn the Source CodeIn the Evolution
  • 49. Design problemsThe most common design problems result from code that isUnclear & complicated Duplicated (code clones) 49
  • 50. Code Smells (if it stinks, change it)A code smell is a hint that something has gone wrongsomewhere in your code.Duplicated CodeLong MethodLarge ClassLong Parameter ListDivergent ChangeShotgun SurgeryFeature Envy... 50
  • 51. How to detect?Measure and visualize quality aspects of the currentimplementation of a system Source code metrics and structuresMeasure and visualize quality aspects of the evolution of asystem Evolution metrics and structuresUse Polymetric Views 51
  • 52. Polymetric ViewsA combination of metrics and softwarevisualization Entity Visualize software using colored rectangles for the entities and edges for the relationships Relationship Render up to five metrics on one node: Size (1+2) Color (3) Position (4+5) X Coordinate Y Coordinate Color tone Height Width 7 52
  • 53. Smell 1: Long MethodThe longer a method is, the more difficult it is to understandit.When is a method too long? Heuristic: > 10 LOCs (?)How to detect? Visualize LOC metric values of methods “Method Length Distribution View” 53
  • 54. Method Length Distribution Metrics: Boxes: Methods Width: LOC Position-Y: LOC Sort: LOC 54
  • 55. Smell 2: Switch StatementProblem is similar to code duplication Switch statement is scattered in different placesHow to detect? Visualize McCabe Cyclomatic Complexity metric to detect complex methods “Method Complexity Distribution View” 55
  • 56. Method ComplexityMetrics:Boxes: MethodsPosition-X: LOCPosition-Y: MCCSort: - 56
  • 57. More info on Detection StrategiesObject-Oriented Metrics in PracticeMichele Lanza and Radu Marinescu, Springer 2006 57
  • 58. Tool for Smell DetectioninCode 58
  • 59. Problem DetectionIn the Source CodeIn the Evolution
  • 60. Understanding EvolutionChanges can point to design problems “Evolutionary Smells”But Overwhelming complexity How can we detect and understand changes?Solutions The Evolution Matrix The Kiviat Graphs 60
  • 61. Visualizing Class EvolutionVisualize classes as rectangles using forwidth and height the following metrics: Foo NOM (number of methods) NOA (number of attributes) BarThe Classes can be categorized accordingto their “personal evolution” and to their“system evolution”-> Evolution Patterns 61
  • 62. The Evolution Matrix Removed Classes Last VersionFirst Version Added Classes Major Leap Growth Stabilisation TIME (Versions) 62
  • 63. Evolution Patterns & SmellsDay-fly (Dead Code)PersistentPulsar (Change Prone Entity)SupernovaWhite Dwarf (Dead Code)Red Giant (Large/God Class)Idle (Dead Code) 63
  • 64. Persistent / Dayfly Dayflies: Exists during only one or two versions. Perhaps Persistent: Has the same an idea which was lifespan as the whole tried out and then system. Part of the dropped. original design. Perhaps holy dead code which no one dares to remove. 64
  • 65. Pulsar / Supernova Pulsar: Repeated Modifications make it grow and shrink. System Hotspot: Every System Version requires changes. Supernova: Sudden increase in size. Possible Reasons: • Massive shift of functionality towards a class. • Data holder class for which it is easy to grow. • Sleeper: Developers knew exactly what to fill in. 65
  • 66. White Dwarf / Red Giant / IdleWhite Dwarf: Lost the functionality it had and now trundles along withoutreal meaning. Possibly dead code -> Lazy Class.Red Giant: A permanent god (large) class which is always very large.Idle: Keeps size over several versions. Possibly dead code,possibly good code. 66
  • 67. Summary Problem DetectionDesign Problems Result from duplicated, unclear, complicated source code -> Code SmellsDetection heuristics and Polymetric Views to detect code andevolution smells 67
  • 68. ConclusionsObject-Oriented Re-engineering Patterns Set of best practices to re-engineering software systemsModule Capture and Reverse Engineering Understand the design and implementation of software systemsProblem Detection Heuristics to detect Bad Smells in the source code and evolution of software systemsNext Step Add tests and refactor detected problems 68
  • 69. Reading material Object-Oriented Reengineering Patterns Serge Demeyer, Stephane Ducasse, and Oscar Nierstrasz free copy from: Working Effectively with Legacy Code Michael Feathers, Prentice Hall, 1 edition, 2004Refactoring to PatternsJoshua Kerievsky, Addison-Wesley Professional, 2004 69
  • 70. Additional reading Agile Software Development: Principles Patterns, and Practices Robert C. Martin, Prentice Hall Object-Oriented Design Heuristics Arthur J. Riel, Prentice Hall, 1 edition, 1996Refactoring: Improving the Design of Existing CodeMartin Fowler, Addison-Wesley Professional, 1999 70