This document summarizes the results of a large-scale empirical study on the relationship between build systems and build maintenance activity. The study analyzed over 800,000 open source projects to compare how different build technologies (e.g. Make, Autotools, Maven) affect build churn, source code coupling, and authorship over time. The key findings are that framework-based build systems tend to have higher build churn, tighter source code coupling, and decreasing authorship as projects migrate to more advanced technologies.

1. A Large-Scale Empirical Study of
the Relationship between Build
Technology and Build Maintenance
Shane Meiyappan Ahmed E. Bram Audris
McIntosh Nagappan Hassan Adams Mockus

2. What is a build system?
Source
code
2

3. What is a build system?
Source Deliverable
code
2

4. Build systems describe how sources
are translated into deliverables
.c .o .a
.cc .o .exe .deb
.tex .dvi .pdf
3

5. 1) Listen for 2) Build
changes changes
Continuous
Integration:
Enabled by
the build
system
4) Publish 3) Execute
results tests
4

6. The Build “Tax”
“...nothing can be said
to be certain, except
death and taxes”
- Benjamin Franklin
Up to 27% of source
changes require build
changes, too!
An Empirical Study of
Build Maintenance Effort
S. McIntosh, B. Adams, T. H. D.
Nguyen,Y. Kamei, A. E. Hassan
[ICSE 2011]
5

7. Teams even migrate to different build technologies
6

8. ...But does build technology
really influence build
maintenance activity?
7

9. Low- Abstraction- Framework-
Level Based Driven
File-level Low-level scripts Assumes standard
dependencies are generated from project layout and
explicitly declared project metadata dependencies
Make
Rake
Autotools
Dependency Management
Automatically manages 3rd-
party API dependencies
8

10. Low- Abstraction- Framework-
Level Based Driven
File-level Low-level scripts Assumes standard
dependencies are generated from project layout and
explicitly declared project metadata dependencies
Make
Rake
Autotools
Dependency Management
Automatically manages 3rd-
party API dependencies
9

11. Low- Abstraction- Framework-
Level Based Driven
File-level Low-level scripts Assumes standard
dependencies are generated from project layout and
explicitly declared project metadata dependencies
Make
Rake
Autotools
Dependency Management
Automatically manages 3rd-
party API dependencies
10

12. Low- Abstraction- Framework-
Level Based Driven
File-level Low-level scripts Assumes standard
dependencies are generated from project layout and
explicitly declared project metadata dependencies
Make
Rake
Autotools
Dependency Management
Automatically manages 3rd-
party API dependencies
11

13. Low- Abstraction- Framework-
Level Based Driven
File-level Low-level scripts Assumes standard
dependencies are generated from project layout and
explicitly declared project metadata dependencies
Make
Rake
Autotools
Dependency Management
Automatically manages 3rd-
party API dependencies
12

14. Mining a Large-
Scale Corpus
Amassing and indexing a
large sample of version
control systems
Towards the census of public
source code history
A. Mockus [MSR 2009]
843,976 source
code repositories!
13

15. Software Forges
Mining a Large-
Scale Corpus
repo.or.cz
Amassing and indexing a
large sample of version
control systems
Towards the census of public
source code history
A. Mockus [MSR 2009]
843,976 source
code repositories!
13

16. Software Forges
Mining a Large-
Scale Corpus
repo.or.cz
Amassing and indexing a
large sample of version
control systems
Towards the census of public Software Ecosystems
source code history
A. Mockus [MSR 2009]
843,976 source
code repositories!
13

17. Software Forges
Mining a Large-
Scale Corpus
repo.or.cz
Amassing and indexing a
large sample of version
control systems
Towards the census of public Software Ecosystems
source code history
A. Mockus [MSR 2009]
843,976 source
Large Projects
code repositories!
13

18. Software Forges
Framework builds
tend to churn most repo.or.cz
Low−level Abstraction Frame Dependency
200
Build churn rate
100
0
Ant Jam Makefile Rake SCons Autotools CMake Maven Bundler Ivy
14

19. Source code coupling is Software Ecosystems
tightest with framework
and abstraction
1.2
1.0
0.8
0.6
0.4
0.2
−0.2
Low-level Framework Low-level Abstraction Low-level Abstraction
Apache − Ant vs. Maven Debian − Make vs. Autotools GNU − Make vs. Autotools
15

20. Migration
Large Projects
helps! KDE
1.00
Authorship
Autotools
0.75 decreases
Build author ratio
●●
●● ●●
●●● ●●●
●●
0.50 ●● ● ●
● ●
●
● ● ●
●
●● ●● ●
●
● ●
● ● ●
●●●● ●
●●
●● ● ●
● ●●●●●●● ●
●
●● ●●●●●
0.25 ●●●
● ●
●
●
●●●●●
●●
●●● ●●
●●●
0.00
2000 2004 2008 2012
16

21. Migration
Large Projects
helps!
KDE
1.00
Authorship
Autotools
0.75 decreases
Build author ratio
●
● ●●
●● ●
●● ● ●●
●●●● ●●
●
● ●
0.50 ● ●
●
● ●●
●●
●● ●
●● ●
●●●● ●
●● ●
● ●● ● ●
●●●●●●● ●
● ● ●
● ●
● ●● ●
0.25 ●
●●
●
●
●
0.00 ●●●●●
●●●●●
●●●●●
2000 2004 2008 2012
Teams dedicate build
experts to help cope
with the build tax 16