Improving Program Comprehension and Recommendation Systems using Developers' Context

1. Ph.D. Defense
Improving Program Comprehension and
Recommendation Systems using Developers’
Context
Z´ephyrin Soh
Supervised by:
Dr. Yann-Ga¨el Gu´eh´eneuc, Dr. Giuliano Antoniol, Dr. Foutse Khomh
Polytechnique Montr´eal, Canada
December 8, 2015
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 1/40

2. Outline
1 Context & Motivations
2 Problems & Thesis Statement
3 Contribution 1: Quality of ITs
4 Contribution 2: Recommendation Systems
5 Contribution 3: Developers’ Effort
6 Contribution 4: Program Exploration
7 Conclusion
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 2/40

3. Context & Motivations
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 3/40

4. Context & Motivations
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 3/40

5. Context & Motivations
Maintenance cost increases [1]
*Source:http://cvmountain.com *Source:https://www.leaseweb.com
1981 1999
[1] Stephen R. Schach, Object-Oriented and Classical Software Engineering. Eighth
Edition McGraw-Hill, New York, 2011, pp. 11
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 4/40

6. Context & Motivations
Typical Scenario for maintenance using ITS and SCR
Stakeholder
(user/developer)
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 5/40

7. Context & Motivations
Typical Scenario for maintenance using ITS and SCR
Stakeholder
(user/developer)
ITS
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 5/40
Maintenance task = change request (bug
or enhancement)

8. Context & Motivations
Typical Scenario for maintenance using ITS and SCR
Stakeholder
(user/developer)
ITS
Triaging
Paul Mary Sylvia
Developers
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 5/40

9. Context & Motivations
Typical Scenario for maintenance using ITS and SCR
Workspace
Stakeholder
(user/developer)
ITS
Triaging
Workspace
(+Mylyn)
Paul Mary Sylvia
Developers
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 5/40
Optionally use a monitoring tool to collect
logs
Mandatory in the policy of some projects

10. Context & Motivations
Typical Scenario for maintenance using ITS and SCR
Workspace
Stakeholder
(user/developer)
ITS
Triaging
Workspace
(+Mylyn)
Paul Mary Sylvia
Developers
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 5/40
Interaction traces or context: Acyivity logs
collected when developers interacted with
program entities through the IDE
Patch: All the changes performed on the
program

11. Context & Motivations
Typical Scenario for maintenance using ITS and SCR
Workspace
Stakeholder
(user/developer)
ITS
Triaging
Workspace
(+Mylyn)
Paul Mary Sylvia
SCR
Commiter/
reviewer
Developers
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 5/40
Task is resolved: “fixed/closed”

12. Context & Motivations
Typical Scenario for maintenance using ITS and SCR
Workspace
Stakeholder
(user/developer)
ITS
Triaging
Workspace
(+Mylyn)
Paul Mary Sylvia
SCR
Commiter
reviewer
Developers
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 5/40
The stakeholders involved in the
development need help, in particular the
one who is in charge of performing
changes to resolve the task

13. Context & Motivations
Workspace
(+Mylyn)
Paul
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 6/40

14. Context & Motivations
Workspace
(+Mylyn)
Paul
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 6/40
Barry Boehm, Software engineering. IEEE Trans. Computers, 1976

15. Context & Motivations
Difficult
Workspace
(+Mylyn)
Paul
Expensive
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 6/40
Ian Sommerville, Software engineering. Pearson, ninth edition, 2011

16. Context & Motivations
Difficult
Workspace
(+Mylyn)
Paul
Expensive
Search
Relate
Collect
Relevant
information
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 6/40
Ko , An exploratory study of how developers seek, relate, and collect relevant
information during software maintenance tasks. IEEE TSE, 2006

17. Context & Motivations
Difficult
Workspace
(+Mylyn)
Paul
~50% time
Expensive
Search
Relate
Collect
Relevant
information
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 6/40
SWEBOK V3.0, Guide of Software Engineering Body of Knowledge. IEEE,
2014, pp. 5-4

18. Context & Motivations
Difficult
Workspace
(+Mylyn)
Paul
~50% time
Expensive
Search
Relate
Collect
Relevant
information
Program
Exploration
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 6/40

19. Context & Motivations
Difficult
Workspace
(+Mylyn)
Paul
~50% time
Expensive
Search
Relate
Collect
Relevant
information
Program
Exploration
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 6/40
How developers explore program to find relevant information to
change impacts their effort (and thus their productivity).

20. Problems & Thesis Statement
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 7/40

21. Problems & Thesis Statement
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 7/40

22. Problems & Thesis Statement
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 7/40

23. Problems & Thesis Statement
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 7/40

24. Problems & Thesis Statement
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 7/40

25. Problems & Thesis Statement
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
No validation
of the
impact of
inacurate ITs
on previous
studies
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 7/40

26. Problems & Thesis Statement
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
No validation
of the
impact of
inacurate ITs
on previous
studies
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 7/40

27. Problems & Thesis Statement
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
No validation
of the
impact of
inacurate ITs
on previous
studies
No empirical
evidence if
complex
change
required
more effort
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 7/40

28. Problems & Thesis Statement
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
No validation
of the
impact of
inacurate ITs
on previous
studies
No empirical
evidence if
complex
change
required
more effort
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 7/40

29. Problems & Thesis Statement
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
No validation
of the
impact of
inacurate ITs
on previous
studies
No empirical
evidence if
complex
change
required
more effort
No systematic
identification
of exploration
strategies
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 7/40

30. Contribution 1: Quality of ITs
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 8/40

31. Contribution 1: Quality of ITs
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 8/40

32. Contribution 1: Quality of ITs
Motivations
ITS
Workspace
(+Mylyn)
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 9/40
ITs are collected in real-work
environment

33. Contribution 1: Quality of ITs
Motivations
ITS
Workspace
(+Mylyn)
Interlease activities Interruptions/Idle times
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 9/40
ITs are collected in real-work
environment
Activities

34. Contribution 1: Quality of ITs
Motivations
ITS
Workspace
(+Mylyn)
Interlease activities Interruptions/Idle times
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 9/40
ITs are collected in real-work
environment
Activities
The time mined from
ITs is the time spent by the
developers performing the
maintenance task.

35. Contribution 1: Quality of ITs
Motivations
Kind = “edit” ⇒ change activity
Edit with not null duration
Intent = time spent in the editor is more productive
Intent vs. actual use
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 10/40

36. Contribution 1: Quality of ITs
Experiment
Are there noises in ITs?
Can we clean noises in ITs?
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 11/40

37. Contribution 1: Quality of ITs
Experiment
Experiment setting
15 participants
Four systems
1 participant = 1 task
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 12/40

38. Contribution 1: Quality of ITs
Experiment
Experiment setting
15 participants
Four systems
1 participant = 1 task
Collected data
Mylyn
Interaction
Traces (RITs)
Videos
Video
Transcription
Video-based
Interaction
Traces (VITs)
Workspace
(+Mylyn)
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 12/40

39. Contribution 1: Quality of ITs
Time-related Noise
e1
e2
e3
ot
d2
it
d3
Time
d1
Global Time (GT) = endDate(e3) - startDate(e1)
Accumulated Time (AT) = d1 + d2 + d3
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 13/40

40. Contribution 1: Quality of ITs
Time-related Noise
Global vs. Accumulated
VITs: Same results (by definition)
RITs: Different results
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 14/40

41. Contribution 1: Quality of ITs
Time-related Noise
RITs vs. VITs
RITs miss on average 6% of the time spent to perform the task.
Difference due to overlaps between events
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 15/40

42. Contribution 1: Quality of ITs
Time-related Noise
RITs
e1
e2
e3
ot
d2
it
d3
Time
d1
Average individual idle times ≈ 30 sec.
Time (RITs) = AT - ot + d
d = it if it < 30 sec.
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 16/40

43. Contribution 1: Quality of ITs
Edit-related Noise
#edit events (duration = 0)
VITs vs. RITs
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 17/40

44. Contribution 1: Quality of ITs
Edit-related Noise
All edits are not real modification of the code
ITs contain about 28% of false edit-events
Feedback from Mylyn community
“... the argument that there is noise in the edit events makes
sense to me.”
“The edit events dont have to be textual edits ...”
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 18/40

45. Contribution 1: Quality of ITs
Cleaning ITs
Noises are prevalents in ITs
Can we clean noises in ITs?
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 19/40

46. Contribution 1: Quality of ITs
Cleaning ITs
Threshold-based approach
Kind = « edit »
No change Code change
Time <= 24s Time > 24s
Double click
open
Static navigation
(F3)
From search view others
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 20/40

47. Contribution 1: Quality of ITs
Cleaning ITs
Threshold-based approach
Kind = « edit »
No change Code change
Time <= 24s Time > 24s
Double click
open
Static navigation
(F3)
From search view others
CITs (Cleaned ITs) = RITs + cleaning rules
False edit: P = 93%; R = 64%
True edit: P = 18%; R = 81%
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 20/40

48. Contribution 1: Quality of ITs
Cleaning ITs
Prediction-based approach
NaiveBayes, J48, Random Forest
Best:J48 with over sampling
False edit: P = 98.5%; R = 90%
True edit: P = 31.8%; R = 77.8%
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 21/40

49. Contribution 1: Quality of ITs
Conclusion
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
- Method to (in)validate
assumptions
- Cleaning approaches
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 22/40

50. Contribution 1: Quality of ITs
Conclusion
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
- Method to (in)validate
assumptions
- Cleaning approaches
- Do not make these
assumptions anymore
- Use our approach to
clean ITs
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 22/40

51. Contribution 2: Recommendation Systems
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
No validation
of the
impact of
inacurate ITs
on previous
studies
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 23/40

52. Contribution 2: Recommendation Systems
Impact of noises on previous studies
Two studies [2,3]
They made the investigated assumptions
Data and code/scripts available
[2] Ying and Robillard, The influence of the task on programmer bahaviour. ICPC
2011
[3] Lee at al., The impact of view histories on edit recommendations. TSE 2015
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 24/40

53. Contribution 2: Recommendation Systems
Editing Styles
Overview
0 10.5
e1 e2
e = first half if +50% time(e) in first half
Fraction(e = first)
0 to 19% = edit-last
87 to 100% = edit-first
Otherwise = edit-throughout
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 25/40

54. Contribution 2: Recommendation Systems
Editing Styles
Approach
VITs (oracle), RITs, CITs
Evaluation
Precision & recall
RITs
Editing style
VITs Editing style
CITs Editing style
Evaluation
Evaluation
Impact
of noise
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 26/40

55. Contribution 2: Recommendation Systems
Editing Styles
Results
Experiment dataset
Edit-first: →
Edit-last: Recall CITs 22%
Edit-throughout: Precision CITs 4%
On average prediction-based Precision CITs 14% and Recall
CITS 7%
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 27/40

56. Contribution 2: Recommendation Systems
Editing Styles
Results
Experiment dataset
Edit-first: →
Edit-last: Recall CITs 22%
Edit-throughout: Precision CITs 4%
On average prediction-based Precision CITs 14% and Recall
CITS 7%
Bugzilla data set
1 970 ITs for 4 systems
66%-41% same categorisation
34%-59% different categorisation
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 27/40

57. Contribution 2: Recommendation Systems
Recommendation Systems
Same dataset
Same source code
Precision 41% (Threshold-based)
Recall 57% (Prediction-based)
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 28/40

58. Contribution 2: Recommendation Systems
Conclusion
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
No validation
of the
impact of
inacurate ITs
on previous
studies
- Method to (in)validate
assumptions
- Cleaning approaches
- Do not make these
assumptions anymore
- Use our approach to
clean ITs
- Improve
recommendation
systems
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 29/40

59. Contribution 2: Recommendation Systems
Conclusion
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
No validation
of the
impact of
inacurate ITs
on previous
studies
- Method to (in)validate
assumptions
- Cleaning approaches
- Do not make these
assumptions anymore
- Use our approach to
clean ITs
- Improve
recommendation
systems
- Noises bias the
results of previous
studies
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 29/40

60. Contribution 3: Developers’ Effort
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
No validation
of the
impact of
inacurate ITs
on previous
studies
No empirical
evidence if
complex
change
required
more effort
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 30/40

61. Contribution 3: Developers’ Effort
Eclipse bug #188083
Patch #74156
File: 2
LOC : 26
+ 18 LOC
- 8 LOC
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 31/40

62. Contribution 3: Developers’ Effort
Eclipse bug #134884
Patch #94002
File: 2
LOC : 20
+ 19 LOC
- 1 LOC
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 31/40

63. Contribution 3: Developers’ Effort
Complexity of the Changes
Which change is more complex?
vs.
Eclipse bug #134884
Patch #94002
File: 2
LOC : 20
+ 19 LOC
- 1 LOC
Eclipse bug #188083
Patch #74156
File: 2
LOC : 26
+ 18 LOC
- 8 LOC
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 31/40

64. Contribution 3: Developers’ Effort
1 How to estimate the effort spend to provide a patch?
2 Does a complex patch need more effort?
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 32/40

65. Contribution 3: Developers’ Effort
Metrics
Developers’ effort
Time Spend: Total duration spent on all files and their
contents
Cyclomatic complexity: Cyclomatic complexity of the
exploration graph
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 33/40

66. Contribution 3: Developers’ Effort
Metrics
Developers’ effort
Time Spend: Total duration spent on all files and their
contents
Cyclomatic complexity: Cyclomatic complexity of the
exploration graph
Complexity of the changes
Entropy: How much the changes are scattered between
files [1]
Change distance: How much difference between the source
code before the changes and source code after.
[1] A. E. Hassan, Predicting faults using the complexity of code changes, ICSE 2009
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 33/40

67. Contribution 3: Developers’ Effort
Matching
How do we match interactions and patches?
2,408 Interactions histories 3,395 Patches
?
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 34/40

68. Contribution 3: Developers’ Effort
Matching
How do we match interactions and patches?
2,408 Interactions histories 3,395 Patches
?
Assumption: An interaction is matched to a patch (i.e., the patch
is the result of the corresponding interaction) if and only if both
are attached to the same bug report, by the same developer at the
same date (date/hour/minutes).
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 34/40

69. Contribution 3: Developers’ Effort
Matching
Unbalanced matchings
Developers modify files without interacting with them:
Changes not requiring much effort, e.g., propagation of
refactoring
Interactions are not collected when performing the task
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 35/40

70. Contribution 3: Developers’ Effort
Matching
Unbalanced matchings
Developers modify files without interacting with them:
Changes not requiring much effort, e.g., propagation of
refactoring
Interactions are not collected when performing the task
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 35/40

71. Contribution 3: Developers’ Effort
Matching
Unbalanced matchings
Developers modify files without interacting with them:
Changes not requiring much effort, e.g., propagation of
refactoring
Interactions are not collected when performing the task
F6F4
F5F3
F2
F1
F7
F3
F2
F1
F8
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 35/40

72. Contribution 3: Developers’ Effort
Matching
Unbalanced matchings
Developers modify files without interacting with them:
Changes not requiring much effort, e.g., propagation of
refactoring
Interactions are not collected when performing the task
F6F4
F5F3
F2
F1
F7
F3
F2
F1
F8
F7
F6
F5
F9
F2
F3
F1
F4
F8
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 35/40

73. Contribution 3: Developers’ Effort
Matching
Unbalanced matchings
Developers modify files without interacting with them:
Changes not requiring much effort, e.g., propagation of
refactoring
Interactions are not collected when performing the task
F6F4
F5F3
F2
F1
F7
F3
F2
F1
F8
F7
F6
F5
F9
F2
F3
F1
F4
F8
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 35/40

74. Contribution 3: Developers’ Effort
Results
Effort vs. complexity of the changes
1028 matchings and 217 unbalanced matchings
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 36/40

75. Contribution 3: Developers’ Effort
Results
Effort vs. complexity of the changes
1028 matchings and 217 unbalanced matchings
Developers do not necessary spend more effort on tasks
requiring more complex changes
Time (sec.)
Cyclomatic Complexity
Cyclomatic Complexity
Time (sec.)
Entropy
Entropy
Change distance
Change distance
0.16
0.27
0.31
0.33
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 36/40

76. Contribution 3: Developers’ Effort
Additional Files
Additional Files
Exploring files that should not be modified
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 37/40

77. Contribution 3: Developers’ Effort
Additional Files
Additional Files
Exploring files that should not be modified
Significantly relevant files vs. additional (useful and
accidental) files
F4
F1
F2
F5
F6
F3
F7
F9F8
62% 38%
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 37/40

78. Contribution 3: Developers’ Effort
Additional Files
Additional Files
Exploring files that should not be modified
Significantly relevant files vs. additional (useful and
accidental) files
Effort vs. number of additional files: 0.63 (time) and 0.82
(cyclomatic complexity)
F4
F1
F2
F5
F6
F3
F7
F9F8
62% 38%
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 37/40

79. Contribution 3: Developers’ Effort
Conclusion
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
No validation
of the
impact of
inacurate ITs
on previous
studies
No empirical
evidence if
complex
change
required
more effort
- Method to (in)validate
assumptions
- Cleaning approaches
- Do not make these
assumptions anymore
- Use our approach to
clean ITs
- Improve
recommendation
systems
- Noises bias the
results of previous
studies
- Improve
knowledge
- Do not use surrogate
effort with the result
- New feature location
methods and tools
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 38/40

80. Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 39/40
Perfect
equality
0 1
Maximum
inequality
RE
UE RE
Use F-Measure to maximize both precision and recall
0.4

81. Conclusion
We can collect and use accurate interaction traces to assess
developers’ effort, understand how developers’ spent their effort,
and assess their program exploration strategies during
maintenance and evolution activities.
No investigation of
the quality of ITs
No validation
of the
impact of
inacurate ITs
on previous
studies
No empirical
evidence if
complex
change
required
more effort
No systematic
identification
of exploration
strategies
- Method to (in)validate
assumptions
- Cleaning approaches
- Do not make these
assumptions anymore
- Use our approach to
clean ITs
- Improve
recommendation
systems
- Noises bias the
results of previous
studies
- Improve
knowledge
- Do not use surrogate
effort with the result
- New feature location
methods and tools
- Improve
knowledge
- Good strategy
- Use strategy to
guide developers
Z´ephyrin Soh (Polytechnique Montr´eal) Context to improve comprehension and recommendation 40/40