This document summarizes a study on the evolution of build systems. The study analyzed two open source projects to understand how build systems change over time at both high and fine-grained levels. At a high level, the size and complexity of build systems were found to evolve similarly to source code, and build times tended to increase with project age. At a fine-grained level, build systems consisted of around 9% of total files, experienced churn on par with source code, and changes typically added or deleted 3-4 lines. Further, up to 27% of development tasks required build changes, and build maintenance work was found to be dispersed across most team members rather than concentrated in a small group.

1. Studying the Evolution
of Build Systems
Shane McIntosh
Queen’s University

2. What is the build system?
2

3. Build systems help
practitioners
3
OK Testers
Developers
Managers

4. Build systems require 12%
of a developer’s
time (on average)
4
Kumfert, G., and Epperly, T.
Software in the DOE: The
Hidden Overhead of the “Build”

5. Address Bar Search Bar
5
4 months before
the issue was fixed!

6. Research hypothesis
Build system maintenance imposes a significant
degree of overhead on software development
6

7. Studied projects
PLplot
7
>30 MLOC!

8. Thesis Overview
1.0 1.1 2.0
1.High level
evolution
analysis
2. Fine-grained
evolution analysis
3. Maintenance
Overhead analysis
8
Acceptance Ratio:
16/51 = 31%
1.0 2.0
Acceptance Ratio:
62/441 = 14%
1.0 2.0

9. High level evolution
1.Code analysis: Do the static size and
complexity of source code and build
system evolve similarly?
2.Runtime analysis: Does the build-time
complexity evolve?
9
1.0 1.1 2.0

10. Our approach
10

11. Code analysis
(Static)
(Static)
1.Do the static size and complexity of
source code and build system evolve
similarly?
2.Does the build-time complexity evolve?
11
1.0 1.1 2.0

12. 12
ANTLR
Abstraction
Module
Restructuring
Module
Restructuring
Source & Build Change Together!

13. Runtime analysis
(Dynamic)
1.Do the static size and complexity of
source code and build system evolve
similarly?
2.Does the build-time complexity evolve?
13
1.0 1.1 2.0

14. Libs built
from source
Build length increases over time
14

15. High level evolution
results
1.Do the static size and complexity of source
code and build system evolve similarly?
2.Does the build-time complexity evolve?
15
Yes, but events may impact
build and source differently.
Yes, length tends to grow as
projects age.

16. Release granularity is
too coarse!
1 2 31.1 1.21.1.1 1.1.2 1.1.3
Fixed bug #1234 Added feature #1212 Issue #4444:
restructured module16

17. Fine-grained evolution
1.How many files does a typical build system
consist of?
2.How much does a typical build system
churn?
3.How large are typical build system
changes?
17
1.0 2.0

18. Revision tagging
1.1.1 1.1.2
Fixed bug #1234 Added feature #1212 Issue #4444:
Restructured module18

19. Revision tagging
Fixed bug #1234
19

20. Size of the build system
1.How many files does a typical build system
consist of?
2.How much does a typical build system
churn?
3.How large are typical build system
changes?
20
1.0 2.0

21. Build accounts for 9%
of files (median)
21

22. Build system rate of
change
1.How many files does a typical build system
consist of?
2.How much does a typical build system
churn?
3.How large are typical build system
changes?
22
1.0 2.0

23. High churn may result
in post-release bugs!
Normalized churn is on
par with source code
23

24. Size of build changes
1.How many files does a typical build system
consist of?
2.How much does a typical build system
churn?
3.How large are typical build system
changes?
24
1.0 2.0

25. Build changes add and
delete 3-4 lines
Source changes add and
delete 6-11 lines
25

26. 26
Fine-grained evolution
1.How many files does a typical build system
consist of?
2.How much does a typical build system
churn?
3.How large are typical build system changes?
9% of files
On par with source
Adds and deletes 3-4 lines

27. How do projects cope
with build maintenance?
27

28. Build maintenance
overhead
1.How often are build changes required to
complete development tasks?
2.How do projects distribute build
maintenance work?
28
1.0 2.0

29. Logical coupling
29
LC(Source Bld)⇒
= 2 ÷ 4
= 50% 1.0 2.0

30. Logical coupling at
revision level is low!
30
Source Build coupling well below 12% estimation!⇒

31. Work item grouping
1.1.1 1.1.2
Fixed bug #1234 Added feature #1212 Issue #4444:
Restructured module31

32. Work item grouping
1.1.1 1.1.2
Fixed bug #1234 Oops, forgot some
code for bug #1234

33. Work item grouping
Bug #1234
33
Fixed bug #1234 Oops, forgot some
code for bug #1234

34. Task-centric analysis
1.How often are build changes required to
complete development tasks?
2.How do projects distribute build
maintenance work?
34
1.0 2.0

35. Work item coupling to
the build is high
35
Mozilla’s Source Build coupling is very high!⇒

36. Developer-centric
analysis
analysis
1.How often are build changes required to
complete development tasks?
2.How do projects distribute build
maintenance work?
36
1.0 2.0

37. If you produce source code,
do you produce build code?
37

38. Dispersed
Ownership:
Build maintenance is
distributed amongst
most team members
38
Concentrated Ownership:
A small team is
responsible for most
of the build maintenance

39. Build maintenance
overhead
1.How often are build changes required to
complete development tasks?
2.How do projects distribute build
maintenance work?
Up to 27% require build changes
Dispersed or concentrated

40. Potential bias Studied
projects
All
projects
40

41. 41
Questions?