Overview of history, present research and future opportunities in management of economic concerns in software maintenance

1. Application of economic models on
software maintenance
Anh Nguyen Duc
Trial lecture
Trondheim, Norway
April 10th 2015

2. 2
Agenda
o Software maintenance
o Cost models of the maintenance phase
o Economic measures of maintenance activities
o Value driven prioritization of maintenance task

3. 3
Significance of software maintenance
2 to 100 times:
maintenance cost vs.
development cost
Figure from Floris and Harald, 2010

4. 4
Software maintenance types
The totality of activities required to provide cost-
effective support to software

5. 5
Software maintenance process

6. 6
Economic concerns in software
maintenance
Business
level
• Should we continue maintaining the system in the next X
year or buying a new system?
• How much to invest on maintenance for the greatest
earned value (saved cost, increased income)?
Product
level
• How much people need to be assigned for maintaining
the module Y?
• How much does maintenance cost in comparison to
development cost?
Project
level
• How long does it take to fix a post-release software
defect?
• Which features require the least cost but provide the
greatest customer value?

7. 7
Agenda
o Software maintenance
o Cost models of maintenance phase
o Economic measures of maintenance activities
o Value driven prioritization of maintenance task

8. 8
History of parametric cost models
SDC PRICE S Putnam SLIM
COCOMO 81 ESTIMACS
Checkpoint
Belady and Lehman
65 72
77 79 81 83 85
SEER-SEM
97 00
COCOMO versions
Tan and Mookerjee
03 05
95
• 65-85: the search for right parametric forms
• 85-95: advances in size and complexity metrics
• 95-05: proliferation of software development styles
• 05-15: advances in prediction and classification models
Ahn’s SMPEEM
Albrecht FP
Niesink et al.’s MFP
98
year
2015

9. 9
Belady and Lehman (1972)
• Effort: total maintenance effort
• p: productive effort, including analysis, design,
code testing
• d: degree of maintenance team familiarity with the
software
• c: complexity caused by lack of structured design
and document
• K: empirical constant, depends on the environment
Effort= p + K c-d

10. 10
 Effort (person-month) as a function of size (LOC)
Effort = b x Size c
• Organic: small teams develops
software in known
environment (b=2.4, c=1.05)
• Embedded: inflexible and
constrained environment
(b=3.6, c=1.20)
• Semidetached: varying levels
of team experience working
on larger projects (b=3.0,
c=1.12)
Effort = b x Size c x EAF
• Intermediate model
• b, c: calibrated factors
• EAF: effort adjustment factor
COCOMO (1981)

11. 11
Example:
o A ERP module with original size of 10,000 KLOC
o Development effort is 600 PMs
o Estimated 1,500 new and modified KLOC per year
o ACT = 1500/ 10000 = 15% per year
o Maintenance cost of a 10 year project:
= 0.15 x 600 x 10 = 900 PMs
Maintenance effort = ACT x Development effort
• NNL: number of new LOCs
• NML: number of modified LOCs
• NOL: number of original LOCs
Annual change traffic (ACT) = (NNL + NML)/ NOL
COCOMO (1981) – Maintenance cost model

12. 12
COCOMO (1997) – Post architecture level
Application
composition
Early design
Post-architecture
o SF: cost drivers, i.e. development
flexibility, architecture/risk resolution,
team cohesion and process maturity
o EM: 17 effort multipliers, i.e. the
required software reliability, database
size, product complexity, required
reusability, documentation, etc.

13. 13
Function Point based estimation
• Function point (FP) as a consistent and early size
metric
• EFP: enhanced project function point
• ADD, CHG, DEL: added, changed and deleted FPs
• CFP: unadjusted function points added by conversion
• VAFa and VAFb: value adjustment factors of the application
after and before the project
EFP = (ADD + CHG + CFP) × VAFa + (DEL × VAFb)

14. 14
Figure from Boehm et al. 2008
• Repeatable estimations
• Easy to modify input
• Easy to customize and refine formula
Advantages
• Subjective inputs
• Unable to deal with exceptional
conditions
• Mainly designed for waterfall
• Needs historical data for calibration
Disadvantages
COCOMO suite of models (1981-2007)

15. 15
Model calibration
• A systematic adjustment of model parameters to get an
expected output with known inputs
• Proper calibration is important
 COCOMO II 1997 calibration of 83
data points: accuracy within 20% of
actual values in 46% of the times
 COCOMO II 1998 recalibration of161
data points: accuracy within 30% of
actual values in 75% of the times

16. 16
Model evaluation
• Modest predicting results
– “… (estimating in general varies) from as much as 85 - 610 % between
predicated and actual values …” (Kemerer 1993)
– “…best models had MMRE (Mean Magnitude of Relative Error) of 60%,
… too inaccurate to be of any use …” (Jørgensen 1997)
• Combination with other estimation approaches
– Expert judgments, case based reasoning, regression models, Bayesian
network …
• Milestone budget and schedule for project stakeholder
negotiation and expectations management
• Basis of software maintenance planning and control

17. 17
Agenda
o Economic aspects of software maintenance
o Economic measures of maintenance activities
o Cost models of maintenance phase
o Value driven prioritization of maintenance task

18. 18
Value-based software maintenance
• The decision to perform maintenance at any time is based on
cost benefit analysis and economic measures
• Earned-value system
• A set of maintenance tasks
• Planned Value (PV) in
money and time of the
tasks
• Quantification of Earn
Value (EV)

19. 19
Return on Investment (ROI)
Example:
o The benefit of the current system is 250,000 USD
o The benefit of the adapted system is 300,000 USD
o The estimated cost of change: 14,000 USD
o The benefit of the maintenance task is:
300,000 – 250,000 = 50,000 USD
o ROI = (50,000 – 14,000) / 14,000 = 2.57
• Benefit = Value of Changed System - Value of Original System
• Cost = Adjusted size / productivity x Cost driver factors
ROI = (benefit – cost) / cost

20. 20
Net present value (NPV)
• Development phase requires n1 years with a cost of c1 at the
end of each year.
• Operation and maintenance (O&M) phase requires n2 years
with a cost of c2 at the end of each year.
• Cost of capital is at a change rate of r %

21. 21
Management of technical debts
• Metaphor indicating possibly of significant economic
consequences in software maintenance
• Technical debts occur when making the change worked as
quickly as possible with as few resources as possible
Increased productivity, lowered development costs of the
current release
Increased maintenance costs, debt out of controls

22. 22
Management of technical debts
• Principal = cost of paying off the debt
– Maintenance task effort estimation
• Interest benefit of paying off
– Interest probability x interest amount
• Which debts to pay off?
• Applications of technical debts
– Project budget allocation
– Prioritizing maintenance tasks

23. 23
Agenda
o Economic aspects of software maintenance
o Economic measures of maintenance activities
o Cost models of maintenance phase
o Value driven prioritization of maintenance task

24. 24
Focus on low-cost, high-value changes
Effort-to-change
Perceived value
Low High
High
Lowo Technical debt pay-off
o Test case triage
o Defect prioritization
o Feature reduction
o …

25. 25
Deciding customer´s perceived value
• Approaches
– Multi criteria decision analysis (MCDA): Analytic hierarchy
process (AHP), PROMETHEE, etc
– Agile: MoSCoW, planning game, quality house, hundred-
dollar test, etc
• Expert judgment, no model based approach

26. 26
Triage and prediction models
Post-release defect
triage
Issue resolution time
estimation
Modification
request (MR)
History of
delta
Communicatio
n data
Project
context
Regression
+ machine
learning
SIZE &
COMPLEXITY
Change proneness
Defect proneness
Auto task assignment

27. 27
Indicators of change effort
Code metrics
o Cyclomatic complexity
o Object oriented metrics
o Entropy of code changes
o Churn metrics
o Code smell
Modification
request (MR)
History of
delta
Communicatio
n data
Project
context
Regression
+ machine
learning
SIZE &
COMPLEXITY

28. 28
Indicators of change effort
Communication data
o No. of comments
o No. of commenters
o Communication
frequency
Project context
o No. of locations
o No. of developers
o CMMI levels
o Team experience levels
o …
Modification
request (MR)
History of
delta
Communicatio
n data
Project
context
Regression
+ machine
learning
SIZE &
COMPLEXITY

29. 29
Indicator of change effort
Figure from Zhang et al. 2013
Figure from Nguyen-Duc et al. 2011

30. 30
Reflection on model building
• Local vs. global prediction
• Implicit assumptions of models
• Universal indicators of software changes
• Data missing and preprocessing
• Compounding impacts

31. 31
Summary
• Cost-benefit concern of software maintenance for
vendors and buyers
• Measuring economic value of software maintenance
activities
• Limited applicability of traditional maintenance costs
• Integration of code, process, human based metrics in
estimating and triaging maintenance tasks

32. 32
Reference
BOOK
1. Barry Boehm, Software Engineering Economics, Prentice-Hall, 1981
2. Guide to the Software Engineering Body of Knowledge – SWEBOK, IEEE Press,
2004
3. I. Sommerville. Software Engineering, 7th edition, Pearson Education, 2004
4. Shari Lawrence Pfleeger, Joanne M. Atlee, Software Engineering: Theory and
Practice, 3rd edition, Pearson Education, 2006.
5. Stefan Biffl, Aybuke Aurum, Barry Boehm, Hakan Erdogmus, Paul Grünbacher,
Value based software engineering, Springer, 2006

33. 33
Reference
PAPER (1)
• Basili, V., et al. (1996). Understanding and predicting the process of software maintenance release.
Proceedings of the 18th international conference on Software engineering, IEEE Computer Society.
• Boehm, B. W. and R. Valerdi (2008). "Achievements and challenges in cocomo-based software
resource estimation." Software, IEEE 25(5): 74-83.
• Chulani, S., et al. (1999). "Calibrating software cost models using bayesian analysis." IEEE
Transactions on Software Engineering: 573-583.
• Čubranić, D. (2004). Automatic bug triage using text categorization. In SEKE 2004: Proceedings of the
Sixteenth International Conference on Software Engineering & Knowledge Engineering, Citeseer.
• Curtis, B., et al. (2012). "Estimating the Principal of an Application's Technical Debt." IEEE
software(6): 34-42.
• Duc, A. N., et al. (2011). Impact of stakeholder type and collaboration on issue resolution time in
OSS Projects. Open Source Systems: Grounding Research, Springer: 1-16.
• Fioravanti, F. and P. Nesi (2001). "Estimation and prediction metrics for adaptive maintenance effort
of object-oriented systems." Software Engineering, IEEE Transactions on 27(12): 1062-1084.

34. 34
Reference
PAPER (2)
• Granja-Alvarez, J. C. and M. J. Barranco-García (1997). "A method for estimating maintenance cost in
a software project: a case study." Journal of Software Maintenance 9(3): 161-175.
• Jorgensen, M. and M. Shepperd (2007). "A systematic review of software development cost
estimation studies." Software Engineering, IEEE Transactions on 33(1): 33-53.
• Kemerer, C. F. (1987). "An empirical validation of software cost estimation models."
Communications of the ACM 30(5): 416-429.
• Mockus, A., et al. (2003). Understanding and predicting effort in software projects. Proceedings of
the 25th International Conference on Software Engineering, IEEE Computer Society.
• Niessink, F. and H. van Vliet (1997). Predicting maintenance effort with function points. Software
Maintenance, 1997. Proceedings., International Conference on, IEEE.
• Sneed, H. M. (2004). A cost model for software maintenance & evolution. Software Maintenance,
2004. Proceedings. 20th IEEE International Conference on, IEEE.
• Vienneau, R. L. (1995). "The present value of software maintenance." Journal of Parametrics 15(1):
18-36.
• Zhang, H., et al. (2013). Predicting bug-fixing time: an empirical study of commercial software
projects. Proceedings of the 2013 International Conference on Software Engineering, IEEE Press.
•
•
•
•
•
•

35. 35
Q&A