0
How Developers Spend
Their Effort During
Maintenance Activities
Zéphyrin Soh, Foutse Khomh,
Yann-Gaël Guéhéneuc, Giuliano ...
Outline


Introduction



Goal and Benefits



Background



Empirical Study



– RQ1
– RQ2
Conclusion



Threats to...
Introduction


Performing maintenance task = spend a certain
effort to provide some results



Effort:



explore the p...
Motivating Example




Exploration graph
Explored, relevant,
additional files
Motivating Example




Exploration graph
Explored, relevant,
additional files
Motivating Example




Exploration graph
Explored, relevant,
additional files
Motivating Examples





Does the effort spend is proportional to the
results?
What are the factors that may affect deve...
Motivations
How developers' spend their effort may
affect their productivity




Some tasks need less effort while other...
Goal


Main goal
How can we help developers based on
the way they spend their effort?



Specific goals
Understand how d...
Benefits


Better assign tasks to developers



A developer may perform well for some kind of
task e.g., task that need ...
Background
Need detailed information about :


Developers' programming activities
Interactions histories
Recorded by Ecli...
Background




Developers must activate the task to gather
interactions, and disable to stop gathered
interactiosns
XML ...
Empirical Study




Four open-source projects : ECF, Mylyn, PDE,
Eclipse Platform
RQ1 : Does the complexity of the imple...
Empirical Study


RQ2 : How developers spend their effort?

What are the factors affecting developers’ effort?
Bug severi...
RQ1 : Effort vs. Complexity of the
implementation


Does the task that require complex implementation
require more effort...
Developers' effort


Time Spend


Total duration spent on all files and their contents

f1(2min), f2.m1(3min), f3.field1...
Complexity of the Implementation


Entropy


How much the changes are scarttered between files

Change 5 LOC : on 1 file...
RQ1 : Approach





Identify the patch that is the result of a given interaction
Ensure that the changes in the patch ar...
RQ1 : Approach
A patch is the result of the corresponding interaction if and
only if both are attached to the same bug rep...
RQ1 : Results
Matching Unbalanced
ECF

15

2

Mylyn

663

122

PDE

132

27

Platform

218

66

1,028

217

Total


After...
RQ1 : Results




Spearman correlation Effort vs. Complexity of the
implementation (0.16 to 0.33)
Developers do not nece...
How developers spend their effort?




Additional files may increase the developers’ effort and
decrease their productiv...
What are the factors affecting
developers’ effort?


Bug severity



Developers' experience
Bug severity

?
?

Developer...
Effort vs. Bug Severity




The effort spent to perform easy tasks must be
different to the effort spent to perform more...
Effort vs. Bug Severity




Bug severity indicates the complexity of the
implementation
Bug severities that required les...
Effort vs. Bug severity


The relation between developers’ effort and bug
severities is project dependant

Complexity of ...
Effort vs. Experience




An experienced developer would spend less effort
compared to inexperienced developers
Approach...
Effort vs. Experience


Experience metrics
–
–

#Files (NF) that the developer changed before

–


#Bugs (NB) that the d...
Effort vs. Experience


More experience when the significantly relevant files for a
given task have already been used in ...
Effort vs. Experience




No consensus about the benefits of experience on the
effort spend
The experience may reduce th...
Effort vs. Experience




No benefits of experience
When a program evolves, developers increasingly perform
tasks on the...
Conclusion
Complexity of the
implementation

Bug severity
Project
dependent

Effort
Developers'
Experience

Additional
Fil...
Threats to Validity


Construct validity : Mismatching (the patch is not the result of
the interaction)
–
–



Remove un...
Thanks for your attention!

Complexity of the
implementation

Bug severity
Project
dependent

Effort
Developers'
Experienc...
Upcoming SlideShare
Loading in...5
×

130705 zephyrin soh - how developers spend their effort during maintenance activities

79

Published on

Software maintenance, developers efforts, exploration strategies

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
79
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "130705 zephyrin soh - how developers spend their effort during maintenance activities"

  1. 1. How Developers Spend Their Effort During Maintenance Activities Zéphyrin Soh, Foutse Khomh, Yann-Gaël Guéhéneuc, Giuliano Antoniol École Polytechnique Montréal July, 5th 2013
  2. 2. Outline  Introduction  Goal and Benefits  Background  Empirical Study  – RQ1 – RQ2 Conclusion  Threats to Validity
  3. 3. Introduction  Performing maintenance task = spend a certain effort to provide some results  Effort:  explore the program  find relevant program entities  understand and make changes on the program entities Result:    Changes made on source code Provide as patch or commit in repository
  4. 4. Motivating Example   Exploration graph Explored, relevant, additional files
  5. 5. Motivating Example   Exploration graph Explored, relevant, additional files
  6. 6. Motivating Example   Exploration graph Explored, relevant, additional files
  7. 7. Motivating Examples   Does the effort spend is proportional to the results? What are the factors that may affect developers' effort?
  8. 8. Motivations How developers' spend their effort may affect their productivity   Some tasks need less effort while others need more effort Some developers may take comparatively less effort than others : for a task of equal complexity  to provide the results of equal size ==> Important to know why these differences 
  9. 9. Goal  Main goal How can we help developers based on the way they spend their effort?  Specific goals Understand how developers spend their effort  Study whether there are factors which can be influenced by tooling ==> Propose such tooling 
  10. 10. Benefits  Better assign tasks to developers  A developer may perform well for some kind of task e.g., task that need to change a method vs. add new method Refine recommendation systems  Be careful when recommending effort consuming entities that are not relevant to a task Guide developers during program exploration  
  11. 11. Background Need detailed information about :  Developers' programming activities Interactions histories Recorded by Eclipse Mylyn plugin ==> opportunity to estimate developers' effort  The changes made to address a task Patches : contains the source code before and after the changes
  12. 12. Background   Developers must activate the task to gather interactions, and disable to stop gathered interactiosns XML files – How developers explore a program entities – The time spend on each entity
  13. 13. Empirical Study   Four open-source projects : ECF, Mylyn, PDE, Eclipse Platform RQ1 : Does the complexity of the implementation of a task reflects developer’s effort? ? Effort Complexity of the implementation
  14. 14. Empirical Study  RQ2 : How developers spend their effort? What are the factors affecting developers’ effort? Bug severity ? ? Developers' Experience Effort ? Additional Files
  15. 15. RQ1 : Effort vs. Complexity of the implementation  Does the task that require complex implementation require more effort? –  Sometimes, a simple implementation can also require a lot of effort Effort : Interactions data    Time spend Cyclomatic complexity Complexity of the implementation : Patches data  Entropy  Change distance
  16. 16. Developers' effort  Time Spend  Total duration spent on all files and their contents f1(2min), f2.m1(3min), f3.field1(1min) ==> 6min   The more a task takes time, the more the effort is spent to understand and perform changes. Cyclomatic complexity  Cyclomatic = edges – vertex + connected_components  connected_components = 1 The more is the cyclomatic complexity of the exploration graph, the more developers' spend effort to explore the program.
  17. 17. Complexity of the Implementation  Entropy  How much the changes are scarttered between files Change 5 LOC : on 1 file vs. on 5 files   The more the entropy is high, the more the changes are scattered between files, thus complex implementation Change distance  How much is the difference between the source code before the changes and source code after the changes Add new LOC vs. rename a variable in a LOC  The greater is the difference between the old and the new code (i.e., more change distance), the more the change is complex
  18. 18. RQ1 : Approach   Identify the patch that is the result of a given interaction Ensure that the changes in the patch are the results of the effort spend in the interaction ==> Use working date and modifications dates  When developers create a patch?  Before disable the task : 26.04%  After disable the task : 73.95%  delta(t) ~ 12 min ==> Cannot automatically match based on these dates
  19. 19. RQ1 : Approach A patch is the result of the corresponding interaction if and only if both are attached to the same bug report, by the same developer at the same date and time.  Unbalanced matchings  Files are changed but not explored ==> refactoring (propagation) or interruption period  no common files between the interaction and the patch ==> Changes without activited the task The patch is not the result of the interaction
  20. 20. RQ1 : Results Matching Unbalanced ECF 15 2 Mylyn 663 122 PDE 132 27 Platform 218 66 1,028 217 Total  After finding where and how to perform the task, developers' disable the task before performing the changes
  21. 21. RQ1 : Results   Spearman correlation Effort vs. Complexity of the implementation (0.16 to 0.33) Developers do not necessary spend more effort on tasks requiring more complex implementations ==> some of the effort spend by developers on a task do not materialise in the patch? Complexity of the implementation Effort
  22. 22. How developers spend their effort?   Additional files may increase the developers’ effort and decrease their productivity How much developers used additional files? – –  Jaccard similarity between the significantly relevant files and explored files On average, developers use about 62% of additional files. Developers who explore a large number of additional files spend more effort to perform the task Effort Additional Files
  23. 23. What are the factors affecting developers’ effort?  Bug severity  Developers' experience Bug severity ? ? Developers' Experience Effort
  24. 24. Effort vs. Bug Severity   The effort spent to perform easy tasks must be different to the effort spent to perform more complex tasks Approach – Make sure that the bug severity indicates the complexity of the implementation • Kruskall-Wallis test : Bug severity vs. Complexity of the implementation – Kruskall-Wallis test : Bug severity vs. Effort
  25. 25. Effort vs. Bug Severity   Bug severity indicates the complexity of the implementation Bug severities that required less files did not necessarily required less changes
  26. 26. Effort vs. Bug severity  The relation between developers’ effort and bug severities is project dependant Complexity of the implementation Bug severity Project dependent Effort
  27. 27. Effort vs. Experience   An experienced developer would spend less effort compared to inexperienced developers Approach – Compute experience metrics – Correlation : Experience vs. Effort
  28. 28. Effort vs. Experience  Experience metrics – – #Files (NF) that the developer changed before –  #Bugs (NB) that the devloper fixed before #LOC (NLOC) : sum of added and deleted LOC Task1 : f1(2 LOC) and f2(5 LOC) –  First task ==> experience (0, 0, 0) Task2 : – – C1 : Significantly relevant files for Task2 = f2, f3, f4 –  Experience (1 task, 2 files, 7 LOC) C2 : Significantly relevant files for Task2 = f6, …, f9 Experience(C1) vs. Experience(C2)?
  29. 29. Effort vs. Experience  More experience when the significantly relevant files for a given task have already been used in previous tasks. ==> Two kind of experience :  Overall Experience (OE) : task-independent  (task) Relevant Experience (RE) : based on relevant files  C1 : Significantly relevant files for Task2 = f2, f3, f4 – –  OE = 1 task, 2 files, and 7 LOC RE = 1 task, 1 file, and 2 LOC C2 : Significantly relevant files for Task2 = f6, …, f9 – OE = 1 task, 2 files, and 7 LOC – RE = 1 task, 0 file, and 0 LOC
  30. 30. Effort vs. Experience   No consensus about the benefits of experience on the effort spend The experience may reduce the effort in some few cases ==> weak negative correlation between effort and experience  Robbes and Röthlisberger [1] also found weak negative correlation using the number of commits to measure experience ==> one can use the number of task and the number of files changed before to measure developers’ experience [1] R. Robbes and D. Röthlisberger. Using developer interaction data to compare expertise metrics. MSR ’13
  31. 31. Effort vs. Experience   No benefits of experience When a program evolves, developers increasingly perform tasks on the files that they never used before Files never used More files never used Less files never used More files never used
  32. 32. Conclusion Complexity of the implementation Bug severity Project dependent Effort Developers' Experience Additional Files
  33. 33. Threats to Validity  Construct validity : Mismatching (the patch is not the result of the interaction) – –  Remove unbalanced matchings No guarantee that we didn't miss some matchings Conclusion validity – –  No violation of assumptions of the statistical tests No causation : only observations based on correlation and distribution of metrics values Internal validity –  Only Mylyn's interaction histories External validity – Open-source projects
  34. 34. Thanks for your attention! Complexity of the implementation Bug severity Project dependent Effort Developers' Experience Additional Files
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×