4. How often did you ...
... encounter greenfield and non-greenfield software
engineering?
4
5. Why non-greenfield engineering?
Because existing software, often called legacy software, is
valuable
Often business-critical
A huge amount of money has already been invested in it
Has been tested and runs
Does (mainly) what it should do
Would you replace such a system?
5
7. Lehman’s Laws of software evolution
Continuing change
A program that is used in a real-world environment must change, or
become progressively less useful in that environment.
Increasing complexity
As a program evolves, it becomes more complex, and extra resources are
needed to preserve and simplify its structure.
For more information read Lehman and Belady, 1985
7
9. Lehman’s Laws in practice
Existing software Is often modified in an ad-hoc manner (quick
fixes)
Lack of time, resources, money, etc.
Initial good design is not maintained
Spaghetti code, copy/paste programming, dependencies are introduced,
no tests, etc.
Documentation is not updated (if there is one)
Architecture and design documents
Original developers leave and with them their knowledge
9
11. Implications of the results
Software maintenance costs continuously increase
Between 50% and 75% of global software development costs
are spent on maintenance!
Up to 60% of a maintenance effort is spent on understanding
the existing software
11
12. What is your decision?
According to Lehman: “there will always be changes”
hack it?
* duplicated code * first reengineer
* complex conditionals * then implement changes
* abusive inheritance
* large classes/methods
Take a loan on your software Investment for the future
pay back via reengineering paid back during maintenance
12
14. Let’s reengineer
Definition:
“Reengineering is the examination and alteration of a subject
system to reconstitute it in a new form and the subsequent
implementation of the new form.”
[Demeyer, Ducasse, Nierstrasz]
http://scg.unibe.ch/download/oorp/
14
15. Reengineering Life-Cycle
(1) requirement New
analysis Requirements
(3) problem
detection (4) problem
resolution
Designs
(2) model
capture
Code
15
17. Goals of reengineering (2)
Unbundling
Split a monolithic system into parts that can be separately marketed
Performance
“First do it, then do it right, then do it fast”
Design extraction
To improve maintainability, portability, etc.
Exploitation of New Technology
I.e., new language features, standards, libraries, etc.
17
18. In this course, you will learn
Best practices to analyze and understand software systems
(i.e., reverse engineering)
Heuristics and tools to detect shortcomings in the design and
implementation of software systems
18
20. Setting direction patterns
Set direction
Where to start?
Agree on Maxims
Maintain Coordinate
direction direction
Most Valuable First
Appoint a Speak to the
Navigator Round Table What to do? What not to do?
Fix Problems, If It Ain't Broke
Not Symptoms Don't Fix It
Principles & guidelines for
software project management are
How to do it?
especially relevant for reengineering
projects
Keep it Simple
20
22. Most valuable first (2)
Solution: Work on aspects that are most valuable to your
customer
Maximize commitment
Deliver results early
Build confidence
22
24. Most valuable first (4)
How do you tell what is valuable?
Identify your customer
Understand the customer’s business model
Determine measurable goals
Consult change logs for high activity
Play the Planning Game
Fix Problems, not Symptoms
24
27. What is Reverse Engineering and why?
Reverse Engineering is the process of analyzing a subject system
to identify the system’s components and their interrelationships and
create representations of the system in another form or at a higher level
of abstraction [Chikofsky & Cross, ’90]
Motivation
Understanding other people’s code, the design and architecture in order
to maintain and evolve a software system
27
28. First contact patterns
feasibility assessment
System experts (one week time)
Talk with Talk with
developers end users
Chat with the Interview Talk about it
Maintainers during Demo
Software System
Verify what
you hear Read about
Read it Compile it
it
Read All the Code Skim the Do a Mock
in One Hour Documentation Installation
28
29. Pattern: Read all the code in one hour
Problem: Yes, but… the system is so big! Where to start?
29
30. Read all the code in one hour (2)
Solution: Read the code in one hour
Focus on:
Functional tests and unit tests
Abstract classes and methods and classes high in the hierarchy
Surprisingly large structures
Comments
Check classes with high fan-out
Study the build process
30
31. In Java programs focus on
public abstract class Example { public interface IExample {
... ...
} }
/**
public class Test {
* Block comment
...
*/
@Test
public class Example {
public void testExample() {
public void foo() {
...
int x = 1;
}
for (int x=1; i<100; i++) {
}
// do something comment
}
}
}
31
32. First project plan
Project scope (1/2 page)
Description, context, goals, verification criteria
Opportunities
Identify factors to achieve project goals
Skilled maintainers, readable source-code, documentation, etc.
Risks
Identify risks that may cause problems
Absent test-suites, missing libraries, etc.
Record likelihood & impact for each risk
Go/no-go decision, activities (fish-eye view)
32
34. Initial understanding patterns
Top down
Recover
design
Speculate about Design
ITERATION
understand ⇒
higher-level model
Analyze the Study the
Persistent Data Exceptional Entities
Recover Identify
database problems
Bottom up
34
35. Study the exceptional entities
Problem: How can you quickly identify design problems?
Solution: Measure software entities and study the anomalous ones
Visualize metrics to get an overview
Use simple metrics
Lines of code
Number of methods
...
35
36. Example: Exceptional entities
Use simple
Use simple
metrics and
metrics and
layout
layout
algorithms
algorithms.
height colour
(x,y) width
Visualize up
to 5 metrics
per node
36
38. Detailed model capture patterns
Tie Code and Questions Expose the design & make sure it
stays exposed
Expose design
Keep track of
your understanding Refactor to Understand
Expose collaborations
Step through the Execution
Expose contracts Write Tests
to Understand
Look for the Contracts
• Use Your Tools
• Look for Key Methods Expose evolution
• Look for Constructor Calls
• Look for Template/Hook Methods Learn from the Past
• Look for Super Calls
38
39. Refactor to understand
Problem: How do you decipher cryptic code?
Solution: Refactor it till it makes sense
Goal (for now) is to understand, not to reengineer
Hints
Work with a copy of the code
Refactoring requires an adequate test base
If this is missing, “Write Tests to Understand”
39
40. Refactor to understand (cont.)
Guidelines
Rename attributes to convey roles
Rename methods and classes to reveal intent
Remove duplicated code
Replace condition branches by methods
40
41. Learn from the past
Problem: How did the system get the way it is? Which parts are
stable and which aren’t?
Solution: Compare versions to discover where code was removed
Removed functionality is a sign of design evolution
Use or develop appropriate tools
Look for signs of:
Unstable design — repeated growth and refactoring
Mature design — growth, refactoring, and stability
41
42. Examples: Unstable design
Pulsar: Repeated Modifications make it grow and shrink.
System Hotspot: Every System Version requires changes.
42
47. Summary Model Capture
Setting direction patterns to
Set the goals
Find the Go/No-Go decision
Increase commitment of clients and developers
First contact patterns to
Obtain an overview and grasp the main issues
Assess the feasibility of the project
Initial Understanding & Detailed Model Capture
Plan the work … and work the plan
Frequent and short iterations
47
49. Design problems
The most common design problems result from code that is
Unclear & complicated Duplicated (code clones)
49
50. Code Smells (if it stinks, change it)
A code smell is a hint that something has gone wrong
somewhere in your code.
Duplicated Code
Long Method
Large Class
Long Parameter List
Divergent Change
Shotgun Surgery
Feature Envy
...
50
51. How to detect?
Measure and visualize quality aspects of the current
implementation of a system
Source code metrics and structures
Measure and visualize quality aspects of the evolution of a
system
Evolution metrics and structures
Use Polymetric Views
51
52. Polymetric Views
A combination of metrics and software
visualization Entity
Visualize software using colored rectangles for
the entities and edges for the relationships Relationship
Render up to five metrics on one node:
Size (1+2)
Color (3)
Position (4+5)
X Coordinate
Y Coordinate
Color tone
Height
Width
7 52
53. Smell 1: Long Method
The longer a method is, the more difficult it is to understand
it.
When is a method too long?
Heuristic: > 10 LOCs (?)
How to detect?
Visualize LOC metric values of methods
“Method Length Distribution View”
53
55. Smell 2: Switch Statement
Problem is similar to code duplication
Switch statement is scattered in different places
How to detect?
Visualize McCabe Cyclomatic Complexity metric to detect complex
methods
“Method Complexity Distribution View”
55
57. More info on Detection Strategies
Object-Oriented Metrics in Practice
Michele Lanza and Radu Marinescu, Springer 2006
http://www.springer.com/computer/swe/book/
978-3-540-24429-5
57
58. Tool for Smell Detection
inCode
http://www.intooitus.com/inCode.html
jDeodorant
http://java.uom.gr/~jdeodorant/
58
60. Understanding Evolution
Changes can point to design problems
“Evolutionary Smells”
But
Overwhelming complexity
How can we detect and understand changes?
Solutions
The Evolution Matrix
The Kiviat Graphs
60
61. Visualizing Class Evolution
Visualize classes as rectangles using for
width and height the following metrics: Foo
NOM (number of methods)
NOA (number of attributes) Bar
The Classes can be categorized according
to their “personal evolution” and to their
“system evolution”
-> Evolution Patterns
61
62. The Evolution Matrix
Removed Classes Last Version
First Version
Added
Classes
Major Leap
Growth Stabilisation
TIME (Versions)
62
64. Persistent / Dayfly
Dayflies: Exists
during only one or
two versions. Perhaps
Persistent: Has the same an idea which was
lifespan as the whole tried out and then
system. Part of the dropped.
original design. Perhaps
holy dead code which no
one dares to remove.
64
65. Pulsar / Supernova
Pulsar: Repeated Modifications make it grow and shrink.
System Hotspot: Every System Version requires changes.
Supernova: Sudden increase in size. Possible Reasons:
• Massive shift of functionality towards a class.
• Data holder class for which it is easy to grow.
• Sleeper: Developers knew exactly what to fill in.
65
66. White Dwarf / Red Giant / Idle
White Dwarf: Lost the functionality it had and now trundles along without
real meaning. Possibly dead code -> Lazy Class.
Red Giant: A permanent god (large) class which is always very large.
Idle: Keeps size over several versions. Possibly dead code,
possibly good code.
66
67. Summary Problem Detection
Design Problems
Result from duplicated, unclear, complicated source code
-> Code Smells
Detection heuristics and Polymetric Views to detect code and
evolution smells
67
68. Conclusions
Object-Oriented Re-engineering Patterns
Set of best practices to re-engineering software systems
Module Capture and Reverse Engineering
Understand the design and implementation of software systems
Problem Detection
Heuristics to detect Bad Smells in the source code and evolution of
software systems
Next Step
Add tests and refactor detected problems
68
69. Reading material
Object-Oriented Reengineering Patterns
Serge Demeyer, Stephane Ducasse, and Oscar Nierstrasz
free copy from: http://scg.unibe.ch/download/oorp/
Working Effectively with Legacy Code
Michael Feathers, Prentice Hall, 1 edition, 2004
Refactoring to Patterns
Joshua Kerievsky, Addison-Wesley Professional, 2004
69
70. Additional reading
Agile Software Development: Principles Patterns, and Practices
Robert C. Martin, Prentice Hall
Object-Oriented Design Heuristics
Arthur J. Riel, Prentice Hall, 1 edition, 1996
Refactoring: Improving the Design of Existing Code
Martin Fowler, Addison-Wesley Professional, 1999
70