The document investigates self-admitted technical debt (SATD) in build systems, particularly focusing on its characteristics, reasons, and purposes. It highlights that SATD is prevalent in 31% of source code files and classifies it into five main types, while emphasizing the importance of understanding and managing SATD in build specifications. The study finds that developers are receptive to addressing SATD and proposes automated classifiers with promising performance for identifying SATD in build systems.

1. Tao Xiao1, Dong Wang2, Shane Mcintosh3, Hideaki Hata4,
Raula Gaikovina Kula1, Takashi Ishio1, Kenichi Matsumoto1
ESEC/FSE 2022 Journal First
(IEEE Transactions on Software Engineering, 2021)
Characterizing and Mitigating Self-Admitted
Technical Debt in Build Systems

2. // TODO: use false due CAMEL-3826 until root
cause is fixed in hawtdb
INDEX_FACTORY.setDeferredEncoding(false);
o Technical Debt (TD)
o TD could be admitted or annotated as comments by developers. These
comments are called self-admitted technical debt (SATD) [1].
SATD comment
What is Technical Debt and Self-Admitted Technical Debt?
[1] Potdar, A., & Shihab, E. (2014, September). An exploratory study on self-admitted technical debt. In 2014 IEEE International Conference on Software Maintenance and Evolution (pp. 91-100). IEEE.
2
Design choice Expedient in short term Future change more costly

3. o SATD existed in 31% of source code files [1].
o SATD can be classfied into five main types [2].
o SATD changes lead to less future defects than non-SATD changes [3].
o 88% of the practitioners mentioned self-admitting their bad
implementations of code through comments [4].
3
[1] Potdar, A., & Shihab, E. (2014, September). An exploratory study on self-admitted technical debt. In 2014 IEEE International Conference on Software Maintenance and Evolution (pp. 91-100). IEEE.
[2] Maldonado, E. D. S., & Shihab, E. (2015, October). Detecting and quantifying different types of self-admitted technical debt. In 2015 IEEE 7Th international workshop on managing technical debt (MTD) (pp. 9-15). IEEE.
[3] Wehaibi, S., Shihab, E., & Guerrouj, L. (2016, March). Examining the impact of self-admitted technical debt on software quality. In 2016 IEEE 23Rd international conference on software analysis, evolution, and reengineering (SANER) (Vol. 1, pp. 179-188).
IEEE.
[4] Vassallo, C., Zampetti, F., Romano, D., Beller, M., Panichella, A., Di Penta, M., & Zaidman, A. (2016, October). Continuous delivery practices in a large financial organization. In 2016 IEEE International Conference on Software Maintenance and Evolution
(ICSME) (pp. 519-528). IEEE.
SATD is widely studied in source code files.
SATD
Design debt
Defect debt
Documentation
debt
Requirement debt
Test Debt

4. o SATD existed in 31% of source code files [1].
o SATD can be classfied into five main types [2].
o SATD changes lead to less future defects than non-SATD changes [3].
o 88% of the practitioners mentioned self-admitting their bad
implementations of code through comments [4].
4
[1] Potdar, A., & Shihab, E. (2014, September). An exploratory study on self-admitted technical debt. In 2014 IEEE International Conference on Software Maintenance and Evolution (pp. 91-100). IEEE.
[2] Maldonado, E. D. S., & Shihab, E. (2015, October). Detecting and quantifying different types of self-admitted technical debt. In 2015 IEEE 7Th international workshop on managing technical debt (MTD) (pp. 9-15). IEEE.
[3] Wehaibi, S., Shihab, E., & Guerrouj, L. (2016, March). Examining the impact of self-admitted technical debt on software quality. In 2016 IEEE 23Rd international conference on software analysis, evolution, and reengineering (SANER) (Vol. 1, pp. 179-188).
IEEE.
[4] Vassallo, C., Zampetti, F., Romano, D., Beller, M., Panichella, A., Di Penta, M., & Zaidman, A. (2016, October). Continuous delivery practices in a large financial organization. In 2016 IEEE International Conference on Software Maintenance and Evolution
(ICSME) (pp. 519-528). IEEE.
SATD
Design debt
Defect debt
Documentation
debt
Requirement debt
Test Debt
There is a gap in investigating SATD in build systems.
Modern software development has a
broader scope than solely producing source code.

5. 5
[5] McIntosh, S., Adams, B., Nguyen, T. H., Kamei, Y., & Hassan, A. E. (2011, May). An empirical study of build maintenance effort. In 2011 33rd International Conference on Software Engineering (ICSE) (pp. 141-150). IEEE.
Build systems play the critical role in software development.
o Build systems
o How many efforts to maintain build systems?
 Build systems maintenance is 27% overhead
on source code development [5].
DEPENDENCY
Build Files Deliverables
Maintain
Does SATD existed in build systems such as
Maven build systems?

6. 6
Motivating examples: locations, reasons, and purposes.
Location
Purpose Reason
Figure 1. Motivating SATD examples

7. 7
Location
Purpose Reason
Figure 1. Motivating SATD examples
Goal: Understand and Manage SATD in build systems.
1. Establish an understanding of SATD in build specification file
2. Help developers be aware and manage these SATD.

8. 8
Research Design
o Studied Projects
The most popular package manager
and automatic build system for Java
o Research Questions
 RQ1 What are the Characteristics of SATD in Build Systems?
 RQ2 Can Automated Classifiers Accurately Identify the Characteristics
of SATD in Build Systems?
 RQ3 To What Extent can SATD be Removed in Build Systems?

9. [CELLRANGE], 2
[CELLRANGE], 3
[CELLRANGE], 6
[CELLRANGE], 9
[CELLRANGE], 10
[CELLRANGE], 10
[CELLRANGE], 57
[CELLRANGE], 159
[CELLRANGE], 24
Software configuration management
Build organization
Project metadata
Repository configuration
Resource configuration
Multi-directory configuration
Build variables
External dependencies configuration
Plugin configuration
Nine locations are identified. SATD tends to occur in the plugin
configuration and the external dependencies configuration code.
9
RQ1 What are the Characteristics of SATD in Build Systems?
(Location)
<plugin> <instructions>
<!-- <scope>compile</scope>
TODO: Fox up again when
comms bug #1561 is fixed-->
<!-- plug-in jackson 2.x annotation
for compatibility with dropwizard
1.3.1 (a bit ugly but this is the only
way to workaround it)-->
<dependency>
<groupId>com.fasterxml.jackson.co
re</groupId>

10. [CELLRANGE], 17
[CELLRANGE], 2
[CELLRANGE], 7
[CELLRANGE], 2
[CELLRANGE], 17
[CELLRANGE], 20
[CELLRANGE], 11
[CELLRANGE], 12
[CELLRANGE], 40
[CELLRANGE], 4
[CELLRANGE], 5
[CELLRANGE], 47
[CELLRANGE], 67
[CELLRANGE], 16
[CELLRANGE], 27
[CELLRANGE]
No reason
Change propagation
Code smell
Symbol visibility
Compiler configuration
Build break
Licensing
Specify metadata
Recursive call
Post-install dependency resolution
Dependency conflict
Missing dependency
Stale dependency
Build tool limitation
External tool limitation
External library limitation
1
RQ1 What are the Characteristics of SATD in Build Systems?
(Reason)
Limitation
Dependency
Recursive call
Document
Build break
Compiler setting
Code smell
No reason
Change propagation
<!-- plug-in jackson 2.x annotation
for compatibility with dropwizard
1.3.1 (a bit ugly but this is the only
way to workaround it)-->
<dependency>
<groupId>com.fasterxml.jackson.co
re</groupId>
<plugin> <instructions>
<!-- <scope>compile</scope>
TODO: Fox up again when
comms bug #1561 is fixed-->
Nine reasons are identified. The most frequent reasons include the
limitations in tools and libraries, and complexities of dependency
management.

11. 11
RQ1 What are the Characteristics of SATD in Build Systems?
(Purpose)
[CELLRANGE], 9
[CELLRANGE], 13
[CELLRANGE], 79
[CELLRANGE], 95
[CELLRANGE], 110
[CELLRANGE
Silence build warnings
Placeholder for later extension
Document for supoptimal implementation
choice
Warning for future developers
Document workaround
Document for later fix
<!-- plug-in jackson 2.x annotation
for compatibility with dropwizard
1.3.1 (a bit ugly but this is the only
way to workaround it)-->
<dependency>
<groupId>com.fasterxml.jackson.co
re</groupId>
<plugin> <instructions>
<!-- <scope>compile</scope>
TODO: Fox up again when
comms bug #1561 is fixed-->
Six Purposes are identified. With documenting issues to be fixed
later and explaining the rationale for a workaround occurring the
most frequently

12. The auto-sklearn classifier achieves the highest value
of F1-score for SATD reasons and purposes.
12
RQ2 Can Automated Classifiers Accurately Identify the
Characteristics of SATD in Build Systems?
Reason Auto-sklearn NB SVM kNN
Precision 0.72 0.66 0.72 0.63
Recall 0.71 0.66 0.67 0.55
F1-score 0.71 0.65 0.65 0.55
Purpose Auto-sklearn NB SVM kNN
Precision 0.81 0.58 0.76 0.70
Recall 0.80 0.54 0.72 0.68
F1-score 0.79 0.54 0.70 0.66

13. 13
RQ3 To What Extent can SATD be Removed in Build Systems?
<plugin>
<groupId>org.eclipse.xtext</groupId>
<artifactId>xtext-maven-
plugin</artifactId>
<!-- Workaround for broken build from
https://github.com/eclipse/xtext/issue
s/1231#issuecomment-401125569 -->
Identify SATD that contains
resolved bug reports
Submit pull requests/issue
reports to repay SATD
Receive Feedback from
developers

14. Responses of SATD repayments suggest that developers
are receptive and reactive to SATD in build systems.
14
RQ3 To What Extent can SATD be Removed in Build Systems?
Identify SATD that contains
resolved bug reports
Submit pull requests/issue
reports to repay SATD
Receive Feedback from
developers
3
1
3
Submitted pull requests
Accepted Closed No response
3
2
2
Submitted issue reports
Resolved Closed No response

15. 15
Developer Perception Survey
o Q1: Is this an example of technical debt in the build system?
o Q2: Have you personally experienced a similar kind of technical debt?
o Q3: Which reason/purpose do you think for this SATD?
18
2
Gender
Male Prefer not to say
3
2
2
6
2
5
Programming years
5--9 10--14 15--19
20--24 25--29 30 and above
1
1
7
10
Maven editing years
Less than 3 3--5
6--9 10 and above

16. 16
Developer Feedback
6
3
1
Reason
Same label
Second frequent label
Others
5
4
1
Purpose
Same label
Second frequent label
Others
Survery results for Q1 and Q2 Survery results for Q3

17. 17
Developer Feedback
“Ideally I suppose Maven should do
that automatically by picking it up
from other files.’’
Inadequate licensing
information
“We faced similar issues with eclipse
link. Sometimes you just document
that it sucks and move on. There’s not
a ton to be done when the upstream
library is complex, heavily integrated
into your project, still active, and
broken.”
“This situation sucks, if some of these
libraries just don’t have a
(maintained) osgi-ified bundle. This is
one of those cases and there’s not
much you can do aside from bundle it
yourself which can be error prone and
a maintenance nightmare.”
Decisions made by
upstream dependencies
External library limitation
produce maintenance
nightmare

18. 18
Contribution Summary
o A taxonomy of Location, Reason, and Purpose of SATD in build systems is
devised, from a qualitative analysis of 500 comments.
o The performance of our proposed automatic classifier is promising,
achieving an F1-score of 0.71–0.79.
o Developers are receptive and reactive to SATD in build systems. Moreover,
our survey study confirms the important role of SATD in build systems.

19. 19
Conclusion & Future work
o To conclude
First to characterize and mitigate SATD in the Maven build system
and explore the feasibility of training automatic classifiers to identify
SATD characteristics.
Tool support to track and manage SATD backlogs.
o Future work
Expanding coded corpus of SATD comments to other build systems
Further study on duplicate SATD

20. 20