The document discusses the concept of software reengineering, highlighting the importance of addressing legacy software and the need for maintenance due to increasing complexity and design issues. It outlines the reengineering process, goals, and best practices, emphasizing the significance of reverse engineering and problem detection to improve software systems. Additionally, it mentions various tools and methodologies for analyzing and visualizing software code to identify 'code smells' and enhance understandability, modifiability, and maintainability.

1. Devnology Lecture
Software Reengineering
Martin Pinzger
Software Engineering Research Group
Delft University of Technology

2. Greenfield software development
2

3. Non-greenfield software development
?
3

4. How often did you ...
... encounter greenfield and non-greenfield software
engineering?
4

5. Why non-greenfield engineering?
Because existing software, often called legacy software, is
valuable
Often business-critical
A huge amount of money has already been invested in it
Has been tested and runs
Does (mainly) what it should do
Would you replace such a system?
5

6. Why do we (often) start from a mess?
6

7. Lehman’s Laws of software evolution
Continuing change
A program that is used in a real-world environment must change, or
become progressively less useful in that environment.
Increasing complexity
As a program evolves, it becomes more complex, and extra resources are
needed to preserve and simplify its structure.
For more information read Lehman and Belady, 1985
7

8. Evolution of Mozilla source code
8

9. Lehman’s Laws in practice
Existing software Is often modified in an ad-hoc manner (quick
fixes)
Lack of time, resources, money, etc.
Initial good design is not maintained
Spaghetti code, copy/paste programming, dependencies are introduced,
no tests, etc.
Documentation is not updated (if there is one)
Architecture and design documents
Original developers leave and with them their knowledge
9

10. Typical result of such practices
10

11. Implications of the results
Software maintenance costs continuously increase
Between 50% and 75% of global software development costs
are spent on maintenance!
Up to 60% of a maintenance effort is spent on understanding
the existing software
11

12. What is your decision?
According to Lehman: “there will always be changes”
hack it?
* duplicated code * first reengineer
* complex conditionals * then implement changes
* abusive inheritance
* large classes/methods
Take a loan on your software Investment for the future
pay back via reengineering paid back during maintenance
12

13. Lecture Outline
Introduction
Reengineering Process
Module Capture/Reverse Engineering
Problem Detection
Summary and Conclusions
13

14. Let’s reengineer
Definition:
“Reengineering is the examination and alteration of a subject
system to reconstitute it in a new form and the subsequent
implementation of the new form.”
[Demeyer, Ducasse, Nierstrasz]
http://scg.unibe.ch/download/oorp/
14

15. Reengineering Life-Cycle
(1) requirement New
analysis Requirements
(3) problem
detection (4) problem
resolution
Designs
(2) model
capture
Code
15

16. Goals of reengineering
Testability
Understandability
Modifiability
Extensibility
Maintainability
…
16

17. Goals of reengineering (2)
Unbundling
Split a monolithic system into parts that can be separately marketed
Performance
“First do it, then do it right, then do it fast”
Design extraction
To improve maintainability, portability, etc.
Exploitation of New Technology
I.e., new language features, standards, libraries, etc.
17

18. In this course, you will learn
Best practices to analyze and understand software systems
(i.e., reverse engineering)
Heuristics and tools to detect shortcomings in the design and
implementation of software systems
18

19. Model Capture
Reverse Engineering
Setting Direction
First Contact
Initial Understanding
Detailed Model Capture

20. Setting direction patterns
Set direction
Where to start?
Agree on Maxims
Maintain Coordinate
direction direction
Most Valuable First
Appoint a Speak to the
Navigator Round Table What to do? What not to do?
Fix Problems, If It Ain't Broke
Not Symptoms Don't Fix It
Principles & guidelines for
software project management are
How to do it?
especially relevant for reengineering
projects
Keep it Simple
20

21. Pattern: Most Valuable First
Problem: Which problems should you address first?
21

22. Most valuable first (2)
Solution: Work on aspects that are most valuable to your
customer
Maximize commitment
Deliver results early
Build confidence
22

23. Most valuable first (3)
23

24. Most valuable first (4)
How do you tell what is valuable?
Identify your customer
Understand the customer’s business model
Determine measurable goals
Consult change logs for high activity
Play the Planning Game
Fix Problems, not Symptoms
24

25. Most valuable first (5)
Planning Game
25

26. Model Capture
Reverse Engineering
Setting Direction
First Contact
Initial Understanding
Detailed Model Capture

27. What is Reverse Engineering and why?
Reverse Engineering is the process of analyzing a subject system
to identify the system’s components and their interrelationships and
create representations of the system in another form or at a higher level
of abstraction [Chikofsky & Cross, ’90]
Motivation
Understanding other people’s code, the design and architecture in order
to maintain and evolve a software system
27

28. First contact patterns
feasibility assessment
System experts (one week time)
Talk with Talk with
developers end users
Chat with the Interview Talk about it
Maintainers during Demo
Software System
Verify what
you hear Read about
Read it Compile it
it
Read All the Code Skim the Do a Mock
in One Hour Documentation Installation
28

29. Pattern: Read all the code in one hour
Problem: Yes, but… the system is so big! Where to start?
29

30. Read all the code in one hour (2)
Solution: Read the code in one hour
Focus on:
Functional tests and unit tests
Abstract classes and methods and classes high in the hierarchy
Surprisingly large structures
Comments
Check classes with high fan-out
Study the build process
30

31. In Java programs focus on
public abstract class Example { public interface IExample {
... ...
} }
/**
public class Test {
* Block comment
...
*/
@Test
public class Example {
public void testExample() {
public void foo() {
...
int x = 1;
}
for (int x=1; i<100; i++) {
}
// do something comment
}
}
}
31

32. First project plan
Project scope (1/2 page)
Description, context, goals, verification criteria
Opportunities
Identify factors to achieve project goals
Skilled maintainers, readable source-code, documentation, etc.
Risks
Identify risks that may cause problems
Absent test-suites, missing libraries, etc.
Record likelihood & impact for each risk
Go/no-go decision, activities (fish-eye view)
32

33. Model Capture
Reverse Engineering
Setting Direction
First Contact
Initial Understanding
Detailed Model Capture

34. Initial understanding patterns
Top down
Recover
design
Speculate about Design
ITERATION
understand ⇒
higher-level model
Analyze the Study the
Persistent Data Exceptional Entities
Recover Identify
database problems
Bottom up
34

35. Study the exceptional entities
Problem: How can you quickly identify design problems?
Solution: Measure software entities and study the anomalous ones
Visualize metrics to get an overview
Use simple metrics
Lines of code
Number of methods
...
35

36. Example: Exceptional entities
Use simple
Use simple
metrics and
metrics and
layout
layout
algorithms
algorithms.
height colour
(x,y) width
Visualize up
to 5 metrics
per node
36

37. Model Capture
Reverse Engineering
Setting Direction
First Contact
Initial Understanding
Detailed Model Capture

38. Detailed model capture patterns
Tie Code and Questions Expose the design & make sure it
stays exposed
Expose design
Keep track of
your understanding Refactor to Understand
Expose collaborations
Step through the Execution
Expose contracts Write Tests
to Understand
Look for the Contracts
• Use Your Tools
• Look for Key Methods Expose evolution
• Look for Constructor Calls
• Look for Template/Hook Methods Learn from the Past
• Look for Super Calls
38

39. Refactor to understand
Problem: How do you decipher cryptic code?
Solution: Refactor it till it makes sense
Goal (for now) is to understand, not to reengineer
Hints
Work with a copy of the code
Refactoring requires an adequate test base
If this is missing, “Write Tests to Understand”
39

40. Refactor to understand (cont.)
Guidelines
Rename attributes to convey roles
Rename methods and classes to reveal intent
Remove duplicated code
Replace condition branches by methods
40

41. Learn from the past
Problem: How did the system get the way it is? Which parts are
stable and which aren’t?
Solution: Compare versions to discover where code was removed
Removed functionality is a sign of design evolution
Use or develop appropriate tools
Look for signs of:
Unstable design — repeated growth and refactoring
Mature design — growth, refactoring, and stability
41

42. Examples: Unstable design
Pulsar: Repeated Modifications make it grow and shrink.
System Hotspot: Every System Version requires changes.
42

43. Reverse engineering tools/prototypes
X-Ray
http://xray.inf.usi.ch/xray.php
DA4Java
http://swerl.tudelft.nl/bin/view/MartinPinzger/MartinPinzgerDA4Java
Code City
http://www.inf.usi.ch/phd/wettel/codecity.html
43

44. X-Ray
44

45. DA4Java
45

46. CodeCity
46

47. Summary Model Capture
Setting direction patterns to
Set the goals
Find the Go/No-Go decision
Increase commitment of clients and developers
First contact patterns to
Obtain an overview and grasp the main issues
Assess the feasibility of the project
Initial Understanding & Detailed Model Capture
Plan the work … and work the plan
Frequent and short iterations
47

48. Problem Detection
In the Source Code
In the Evolution

49. Design problems
The most common design problems result from code that is
Unclear & complicated Duplicated (code clones)
49

50. Code Smells (if it stinks, change it)
A code smell is a hint that something has gone wrong
somewhere in your code.
Duplicated Code
Long Method
Large Class
Long Parameter List
Divergent Change
Shotgun Surgery
Feature Envy
...
50

51. How to detect?
Measure and visualize quality aspects of the current
implementation of a system
Source code metrics and structures
Measure and visualize quality aspects of the evolution of a
system
Evolution metrics and structures
Use Polymetric Views
51

52. Polymetric Views
A combination of metrics and software
visualization Entity
Visualize software using colored rectangles for
the entities and edges for the relationships Relationship
Render up to five metrics on one node:
Size (1+2)
Color (3)
Position (4+5)
X Coordinate
Y Coordinate
Color tone
Height
Width
7 52

53. Smell 1: Long Method
The longer a method is, the more difficult it is to understand
it.
When is a method too long?
Heuristic: > 10 LOCs (?)
How to detect?
Visualize LOC metric values of methods
“Method Length Distribution View”
53

54. Method Length Distribution
Metrics:
Boxes: Methods
Width: LOC
Position-Y: LOC
Sort: LOC
54

55. Smell 2: Switch Statement
Problem is similar to code duplication
Switch statement is scattered in different places
How to detect?
Visualize McCabe Cyclomatic Complexity metric to detect complex
methods
“Method Complexity Distribution View”
55

56. Method Complexity
Metrics:
Boxes: Methods
Position-X: LOC
Position-Y: MCC
Sort: -
56

57. More info on Detection Strategies
Object-Oriented Metrics in Practice
Michele Lanza and Radu Marinescu, Springer 2006
http://www.springer.com/computer/swe/book/
978-3-540-24429-5
57

58. Tool for Smell Detection
inCode
http://www.intooitus.com/inCode.html
jDeodorant
http://java.uom.gr/~jdeodorant/
58

59. Problem Detection
In the Source Code
In the Evolution

60. Understanding Evolution
Changes can point to design problems
“Evolutionary Smells”
But
Overwhelming complexity
How can we detect and understand changes?
Solutions
The Evolution Matrix
The Kiviat Graphs
60

61. Visualizing Class Evolution
Visualize classes as rectangles using for
width and height the following metrics: Foo
NOM (number of methods)
NOA (number of attributes) Bar
The Classes can be categorized according
to their “personal evolution” and to their
“system evolution”
-> Evolution Patterns
61

62. The Evolution Matrix
Removed Classes Last Version
First Version
Added
Classes
Major Leap
Growth Stabilisation
TIME (Versions)
62

63. Evolution Patterns & Smells
Day-fly (Dead Code)
Persistent
Pulsar (Change Prone Entity)
SupernovaWhite Dwarf (Dead Code)
Red Giant (Large/God Class)
Idle (Dead Code)
63

64. Persistent / Dayfly
Dayflies: Exists
during only one or
two versions. Perhaps
Persistent: Has the same an idea which was
lifespan as the whole tried out and then
system. Part of the dropped.
original design. Perhaps
holy dead code which no
one dares to remove.
64

65. Pulsar / Supernova
Pulsar: Repeated Modifications make it grow and shrink.
System Hotspot: Every System Version requires changes.
Supernova: Sudden increase in size. Possible Reasons:
• Massive shift of functionality towards a class.
• Data holder class for which it is easy to grow.
• Sleeper: Developers knew exactly what to fill in.
65

66. White Dwarf / Red Giant / Idle
White Dwarf: Lost the functionality it had and now trundles along without
real meaning. Possibly dead code -> Lazy Class.
Red Giant: A permanent god (large) class which is always very large.
Idle: Keeps size over several versions. Possibly dead code,
possibly good code.
66

67. Summary Problem Detection
Design Problems
Result from duplicated, unclear, complicated source code
-> Code Smells
Detection heuristics and Polymetric Views to detect code and
evolution smells
67

68. Conclusions
Object-Oriented Re-engineering Patterns
Set of best practices to re-engineering software systems
Module Capture and Reverse Engineering
Understand the design and implementation of software systems
Problem Detection
Heuristics to detect Bad Smells in the source code and evolution of
software systems
Next Step
Add tests and refactor detected problems
68

69. Reading material
Object-Oriented Reengineering Patterns
Serge Demeyer, Stephane Ducasse, and Oscar Nierstrasz
free copy from: http://scg.unibe.ch/download/oorp/
Working Effectively with Legacy Code
Michael Feathers, Prentice Hall, 1 edition, 2004
Refactoring to Patterns
Joshua Kerievsky, Addison-Wesley Professional, 2004
69

70. Additional reading
Agile Software Development: Principles Patterns, and Practices
Robert C. Martin, Prentice Hall
Object-Oriented Design Heuristics
Arthur J. Riel, Prentice Hall, 1 edition, 1996
Refactoring: Improving the Design of Existing Code
Martin Fowler, Addison-Wesley Professional, 1999
70