A CONTROLLED EXPERIMENT IN
ASSESSING AND ESTIMATING
SOFTWARE MAINTENANCE TASKS
Pondicherry University
By:
SADIQUE
NAYEEM
Outline
Motivation and Background
Experiment Design
Results and Explanatory Models
Conclusions
2
MOTIVATION AND
BACKGROUND
3
Software Maintenance
 Software Maintenance
 “Modification of a software product after
delivery to correct faults, to imp...
Maintenance is crucial in software
engineering
 Systems are tightly coupled with their environment
 Environment changes ...
 Maintenance cost is usually 2x to 100x as much as new
development cost (Sommerville, 2006)
K$100 200 300
System 1
System...
For software maintenance, the modeling process
is even
more challenging
 The maintenance effort is affected by a large nu...
Software estimation community has paid
little attention to software maintenance
 Most estimation models regard maintenanc...
Types of Software Maintenance
 Types of Maintenance
Different author proposed different classification of Software
Mainte...
Impact of the types of software
evolution and maintenance.
Ned Chapin, et al., “Types of software evolution and software m...
The purpose of the study
 To assess the size and effort implications
 Effort distribution of three different maintenance...
Research Questions and Hypotheses
 RQ1: Are there any differences in the
productivity of enhancive, corrective, and
deduc...
EXPERIMENTAL
DESIGN
13
Experiment Description
 23 students and 1 senior, computer science major
 Participants worked on tasks individually in t...
Matrices
Independent variable -- type of maintenance (enhancive,
corrective, and reductive)
Dependent variables -- program...
Calculating Maintenance SLOC
Equivalent SLOC = TRCF x AAM




>+
≤−−+
=
=
1,
100
1,])1(1[1( 2
AAFfor
UNFMxSU
AAF
AAFf...
RESULTS AND MODELS
17
Resulted Data
 Each task requires four maintenance activities
 Task comprehension (reading, understanding task requireme...
Effort distribution is different among the
groups (For H2)
 Corrective group spent much
time for code isolation
 twice a...
Productivity
20
Enhancive group produced
1.5 times as many Reductive group deleted and
4 times as many Corrective group mo...
Productivity is significantly different
among the groups (For H1)
Box Plot
 Corrective group has lowest
productivity
 Re...
Participant Time Explanatory Models
M1 Effort = 78.1 + 2.2 * S * EAF
M2 Effort = 43.9 + (2.8*Add + 5.3*Mod + 1.3*Del) * EA...
Model M2 outperforms M1 , M3, M4
i
ii
i
Actual
EstimateActual
MRE
−
=
23
Fig The MRE values obtained by the models in esti...
Threats to Validity
 Internal design
 Groups have imbalanced skills and capability
 Imbalanced complexity of tasks on t...
Conclusions
 Productivity and effort distribution are significantly different
among the maintenance types
 SLOC metrics ...
Upcoming SlideShare
Loading in …5
×

A controlled experiment in assessing and estimating software maintenance tasks

801
-1

Published on

A controlled experiment in assessing and estimating software maintenance tasks

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
801
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • These charts show the effort distributions of four activities in three groups. Four activities are task comprehension, which involves in understanding the task description including asking for clarification, isolation – involves in understanding source code, editing code which is implementing changes, and unit test – testing the code implemented for the change.
  • A controlled experiment in assessing and estimating software maintenance tasks

    1. 1. A CONTROLLED EXPERIMENT IN ASSESSING AND ESTIMATING SOFTWARE MAINTENANCE TASKS Pondicherry University By: SADIQUE NAYEEM
    2. 2. Outline Motivation and Background Experiment Design Results and Explanatory Models Conclusions 2
    3. 3. MOTIVATION AND BACKGROUND 3
    4. 4. Software Maintenance  Software Maintenance  “Modification of a software product after delivery to correct faults, to improve performance or other attributes, or to adapt the product to a modified environment” [IEEE ‘98]  “Changes are made in response to change requirement but the fundamental software structure is stable. 4
    5. 5. Maintenance is crucial in software engineering  Systems are tightly coupled with their environment  Environment changes require changing its software systems  Technologies and requirements are continuously changing  Software systems are outdated quickly  Software systems must be updated and upgraded to maintain their values  Maintenance is important in market competition  New software has more advantages than the existing one  Software system must be upgraded to keep its market share 5
    6. 6.  Maintenance cost is usually 2x to 100x as much as new development cost (Sommerville, 2006) K$100 200 300 System 1 System 2 (Summerville, 2006)New Development Maintenance 400 Development and Maintenance cost6
    7. 7. For software maintenance, the modeling process is even more challenging  The maintenance effort is affected by a large number of factors such as:  Size and types of maintenance work  Team stability  Personnel capabilities  The level of programmer’s familiarity with the system being maintained  Processes and standards in use  Complexity  Technologies  The quality of existing source code and its supporting documentation 7
    8. 8. Software estimation community has paid little attention to software maintenance  Most estimation models regard maintenance estimation as secondary  Models were built using mainly data of new development projects  Use of SLOC metrics for models is inconsistent  Some models use SLOC added, modified, deleted  Others use only SLOC added and modified 8
    9. 9. Types of Software Maintenance  Types of Maintenance Different author proposed different classification of Software Maintenance  Swanson ’76 : Adaptive, Corrective, Perfective  IEEE ’98: all Swanson’s plus Preventive  Chapin et al, 2001: 12 types, 4 clusters: support interface, documentation, software properties, and business rules. The last cluster, includes all activities that alter the business rules of the software :  Enhancive : ‘Did the activities replace, add to, or extend the customer-experienced functionality?’  Corrective : ‘Did the activities fix the customer-experienced functionality or make it more correct?’  Reductive : ‘Did the activities restrict or reduce the customer- 9
    10. 10. Impact of the types of software evolution and maintenance. Ned Chapin, et al., “Types of software evolution and software maintenance,” Jan. 2001 10 Figure 1. Impact of the types of software evolution and maintenance.
    11. 11. The purpose of the study  To assess the size and effort implications  Effort distribution of three different maintenance types  To describe estimation models to predict the programmer’s effort on maintenance tasks.  The focus of study is on enhancive, corrective, and reductive maintenance types according to a maintenance topology proposed by Chapin et al. because they are the ones that change the business rules of the system by adding, modifying, and deleting the source code. 11
    12. 12. Research Questions and Hypotheses  RQ1: Are there any differences in the productivity of enhancive, corrective, and deductive maintenance?  Hypothesis 1 (H1): no difference  RQ2: Are there any differences in the effort distribution among the maintenance types?  Hypothesis 2 (H2): no difference 12
    13. 13. EXPERIMENTAL DESIGN 13
    14. 14. Experiment Description  23 students and 1 senior, computer science major  Participants worked on tasks individually in the lab  Time limit of 7 hour.  Participants were given the original source code, a list of maintenance activities, and a timesheet form.  Participants were required to record time on paper for every activity performed to complete maintenance tasks. Time information includes start clock time, stop clock time, and interruption time measured in minute.  Participants used Visual Studio 2005 for maintenance on Windows XP  UCC as a target program  5188 source statements (logical SLOC) in 20 C++ classes  The program was well-structured and well-commented 14
    15. 15. Matrices Independent variable -- type of maintenance (enhancive, corrective, and reductive) Dependent variables -- programmer’s effort and size of change. The programmer’s effort = total time the programmer spent to work on the maintenance task excluding interruption time. The size of change is measured by three LOC metrics, LOC added, modified, and deleted.
    16. 16. Calculating Maintenance SLOC Equivalent SLOC = TRCF x AAM     >+ ≤−−+ = = 1, 100 1,])1(1[1( 2 AAFfor UNFMxSU AAF AAFforUNFMxSUxAAFAAF AAM TRCF S AAF TRCF = the total SLOC of task-relevant code fragments S = the size in SLOC (added, modified, or deleted) SU = the software understandability(10% to 50%) UNFM = the level of programmer unfamiliarity with the program(0.0 to 1.0) AAF= adaptation adjustment factor AAM= adaptation adjustment multiplier Task-relevant code fragment (TRCF) 16 In this experiment, COCOMO II reuse model is used to determine the equivalent LOC of the maintenance tasks.
    17. 17. RESULTS AND MODELS 17
    18. 18. Resulted Data  Each task requires four maintenance activities  Task comprehension (reading, understanding task requirements, and asking for further clarification)  Code isolation(locating and understanding code segments to be adapted)  Editing code(programming and debugging the affected code)  Unit test(performing tests on the affected code)  Timesheet has 490 activity records, totaling 4621 min.  Total of 909 SLOC added, modified, and deleted 402 added 216 modified 291 deleted 18
    19. 19. Effort distribution is different among the groups (For H2)  Corrective group spent much time for code isolation  twice as much as that of the enhancive group  Enhancive group spent majority of time for editing code  Effort distribution is statistically different among three groups  H2 is rejected 19
    20. 20. Productivity 20 Enhancive group produced 1.5 times as many Reductive group deleted and 4 times as many Corrective group modified. Work done by each group as per the pattern dictated by the task design.
    21. 21. Productivity is significantly different among the groups (For H1) Box Plot  Corrective group has lowest productivity  Reductive group has highest productivity  Productivity between groups are statistically different  H1 is rejected Enhancive Reductive Corrective Productivity(SLOC/Hour) 40 30 20 10 21
    22. 22. Participant Time Explanatory Models M1 Effort = 78.1 + 2.2 * S * EAF M2 Effort = 43.9 + (2.8*Add + 5.3*Mod + 1.3*Del) * EAF Effort = Time spent by the participant on all maintenance tasks EAF = Effort adjustment factor, a product of Programmer capability (PCAP), Language experience (LTEX), and Platform Experience (PLEX) Add, Mod, Del = Equivalent SLOC added, modified, and deleted by the participant, respectively S = Add + Mod + Del R2 = The coefficient of determination values R2 = 0.5 R2 = 0.75 22 R2 = 0.55 R2 = 0.64 M3 Effort = 110.0 + 2.2 * S * EAF M4 Effort = 79.1 + (2.3*Add + 4.6*Mod) * EAF In this experiment, EAF is defined as the multiplicative of programmer capability (PCAP), language and tools experience (LTEX), and platform experience (PLEX). The same rating values for these cost drivers that are defined in the COCOMO II Post-Architecture model is used.
    23. 23. Model M2 outperforms M1 , M3, M4 i ii i Actual EstimateActual MRE − = 23 Fig The MRE values obtained by the models in estimating the effort spent by 24 participants Magnitude of relative error (MRE)
    24. 24. Threats to Validity  Internal design  Groups have imbalanced skills and capability  Imbalanced complexity of tasks on three groups  Incorrect time recorded  Generalizability  Professional programmers are more experienced  Professional programmers are more familiar with the software maintained  Maintenance process is different in industry 24
    25. 25. Conclusions  Productivity and effort distribution are significantly different among the maintenance types  SLOC metrics are relevant factors for estimating effort  Three SLOC metrics (added, modified, and deleted) have different impact on effort  SLOC deleted is an important factor for estimating effort  It is more expensive to modify than to add or delete a statement  Assigning experienced programmers to fixing defect can save up to 40% of effort 25

    ×