1. An Empirical Study of
Build Maintenance Effort
Ahmed E.
Hassan
Bram
Adams
Yasutaka
Kamei
Thanh H.D.
Nguyen
Shane
McIntosh

2. Slides available online!
2
http://goo.gl/yXHHS
http://goo.gl/eICXC

3. Feedback
3
shanemcintosh@acm.org

4. Feedback
3
shanemcintosh@acm.org
Visit my poster
during the break!

5. 4

6. 4

7. 4

8. The Build System Translates
Sources into Deliverables
4Autotools
Make

9. Step 1 - Configuration
5
Features
Autotools

10. Step 1 - Configuration
5
Features
Autotools

11. Step 1 - Configuration
5
Features
Tools
Autotools

12. Step 1 - Configuration
5
Features
Tools
Autotools

13. Step 2 - Construction
6
Prescriptions
Make

14. Step 2 - Construction
6
=
Prescriptions
Make

15. Step 2 - Construction
6
=
Prescriptions
Make

16. Step 2 - Construction
6
=
.cPrescriptions
Make

17. Step 2 - Construction
6
=
.cPrescriptions
Make

18. Step 2 - Construction
6
=
.cPrescriptions
.o
Make

19. Step 2 - Construction
6
=
.cPrescriptions
Dependencies
.o
Make

20. Step 2 - Construction
6
=
.cPrescriptions
Dependencies
.o
Make

21. Step 2 - Construction
6
=
.cPrescriptions
Dependencies
.o
.o.o.o
Make

22. Step 2 - Construction
6
=
.cPrescriptions
Dependencies
.o
.exe
.o.o.o
Make

23. Build code is complex...
7

24. Build Maintenance is a
Nuisance for Developers
8

25. Build Maintenance is a
Nuisance for Developers
8

26. Build Maintenance is a
Nuisance for Developers
8
Autotools
Make

27. Build Maintenance is a
Nuisance for Developers
8
Autotools
Make
Migration to
newer build
technology

28. 9
Build Bugs Even Affect
the End User!

29. Address Bar
9
Build Bugs Even Affect
the End User!

30. Address Bar Search Bar
9
Build Bugs Even Affect
the End User!

31. Address Bar Search Bar
9
Build Bugs Even Affect
the End User!

32. Address Bar Search Bar
9
Build Bugs Even Affect
the End User!

33. Build Systems Require 12%
of a Developer’s
Time (on average)
10
Kumfert, G., and Epperly,T.
Software in the DOE: The
Hidden Overhead of the “Build”

34. Build Systems Require 12%
of a Developer’s
Time (on average)
10
Kumfert, G., and Epperly,T.
Software in the DOE: The
Hidden Overhead of the “Build”
Build maintenance
slows development!

35. 11
Size Evolution Coupling People
Four Dimensional Study of
Build Maintenance
.mk?.c

36. Empirical Study of 10
Systems (>25 MLOC)
PLplot
12

37. Empirical Study of 10
Systems (>25 MLOC)
PLplot
12
PDE Build,ANT,
GNU Autotools, Make

38. Source,Test, or Build?
13
.mk.c.c.ac.dat.h

39. Source,Test, or Build?
13
.mk.c
Test
.c
BuildSource
.ac.dat.h
BuildTestSource

40. 14
Evolution Coupling People
How Large is a Typical
Build System?
.mk?.c
Size

41. The Build Accounts for
9% of All Files (median)
15
0
0.09
0.18
0.27
0.36
16%
31%
12%
8%
3%
5%
10%
12%
1%
7%
% Build files
PLplot

42. 16
Evolution Coupling People
How Large is a Typical
Build System?
.mk?.c
Size
9% of
all files

43. 17
Coupling People
How Does the Build
Evolve?
.mk?.c
Size
9% of
all files
Evolution

44. b.ca.c
File Churn Adjusted by
System Size (# files)
18
Feb 1Jan 1
3 source files
x.c

45. b.ca.c
File Churn Adjusted by
System Size (# files)
18
Feb 1Jan 1
3 source files
x.c

46. b.ca.c
File Churn Adjusted by
System Size (# files)
18
Feb 1Jan 1
3 source files
Source churn
2 ÷ 3 = 67%
x.c

47. b.ca.c .mk
File Churn Adjusted by
System Size (# files)
18
Feb 1Jan 1
3 source files 2 Build specs
Source churn
2 ÷ 3 = 67%
cfgx.c

48. b.ca.c .mk
File Churn Adjusted by
System Size (# files)
18
Feb 1Jan 1
3 source files 2 Build specs
Source churn
2 ÷ 3 = 67%
cfgx.c

49. b.ca.c .mk
File Churn Adjusted by
System Size (# files)
18
Feb 1Jan 1
3 source files 2 Build specs
Source churn
2 ÷ 3 = 67%
Build churn
1 ÷ 2 = 50%
cfgx.c

50. Build File Churn is on Par
with the Source Churn
ML Hibernate−core Eclipse−core Jazz
19
PLplot
GCC Git Linux Mozilla PLplot P
Projects
0.00.20.40.60.81.0
ArgoUML Hibernate Eclipse Jazz GCC Git
Projects
NormalizedChurn
Source
Build
File
020406080100
(%)

51. Build File Churn is on Par
with the Source Churn
ML Hibernate−core Eclipse−core Jazz
19
PLplot
GCC Git Linux Mozilla PLplot P
Projects
0.00.20.40.60.81.0
ArgoUML Hibernate Eclipse Jazz GCC Git
Projects
NormalizedChurn
Source
Build
Build technology
migration
File
020406080100
(%)

52. Line Churn Adjusted by
System Size (# files)
20
x.cb.ca.c
3 source files
Feb 1Jan 1

53. Line Churn Adjusted by
System Size (# files)
20
x.cb.ca.c
3 source files
Feb 1Jan 1
65 4 5

54. Line Churn Adjusted by
System Size (# files)
20
x.cb.ca.c
3 source files
Feb 1Jan 1
65 4
Source Lines per Change list
(5+6+4+5) ÷ 4 = 5
5

55. Line Churn Adjusted by
System Size (# files)
20
x.cb.ca.c
3 source files
Feb 1Jan 1
65 4
Source Lines per Change list
(5+6+4+5) ÷ 4 = 5
Adjust for system size
5 ÷ 3
= 1.67
5

56. Build Line Churn is Higher
than the Source Churn
21
0
0.175
0.35
0.525
0.7
0.52
0.62
0.33
0.37
0.28
0.19
AverageLinesperChangeList(normalizedbysystemsize)
Source
Build

57. 22
Coupling People
How Does the Build
Evolve?
.mk?.c
Size
9% of
all files
Evolution
Build churn
>> source

58. 23
People
How Tightly Coupled are
Build and Source Code?
.mk?.c
Size
9% of
all files
Evolution Coupling
Build churn
>> source

59. Changes, Change Lists,
and Work Items
24
.c .c
Changes
.mk?.c

60. Changes, Change Lists,
and Work Items
24
.c .c
Changes
Change Lists
Fix for bug
#1234
.mk?.c

61. Changes, Change Lists,
and Work Items
24
.c .c
Changes
Change Lists
Fix for bug
#1234
.mk?.c

62. Changes, Change Lists,
and Work Items
24
.c .c .mk
Changes
Change Lists
Fix for bug
#1234
.mk?.c

63. Changes, Change Lists,
and Work Items
24
.c .c .mk
Changes
Change Lists
Fix for bug
#1234
Missed code for
bug #1234
.mk?.c

64. Changes, Change Lists,
and Work Items
24
.c .c .mk
Changes
Change Lists
Fix for bug
#1234
Missed code for
bug #1234
.mk?.c

65. Changes, Change Lists,
and Work Items
24
.c .c .mk
Changes
Change Lists
Fix for bug
#1234
Missed code for
bug #1234
Work items
Bug #1234
.mk?.c

66. Changes, Change Lists,
and Work Items
24
.c .c .mk
Changes
Change Lists
Fix for bug
#1234
Missed code for
bug #1234
Work items
Bug #1234
.mk?.c

67. foo.c
bar.c
Makefile
configure.ac
CL1 CL2 CL3 CL4 CL5
Build
Files
Change Lists
Source
Files
Logical Coupling Example
25
.mk?.c

68. foo.c
bar.c
Makefile
configure.ac
CL1 CL2 CL3 CL4 CL5
Build
Files
Change Lists
Source
Files
Logical Coupling Example
25
.mk?.c

69. foo.c
bar.c
Makefile
configure.ac
CL1 CL2 CL3 CL4 CL5
Build
Files
Change Lists
Source
Files
Logical Coupling Example
25
.mk?.c

70. foo.c
bar.c
Makefile
configure.ac
CL1 CL2 CL3 CL4 CL5
Build
Files
Change Lists
Source
Files
Logical Coupling Example
25
LC(Source Build)
= 2 ÷ 4
= 50%
.mk?.c

71. Change List Coupling is Below
12% Estimate
26
4%
7%7%
3%
5%5%
4%
7%
4%
3%
5%
2%
Source Build
Test Build
.mk?.c
12%

72. Work Item Coupling is High
27
8%
20%
44%
4%
16%
27%
Source Build
Test Build
.mk?.c
12%

73. Work Item Coupling is High
27
8%
20%
44%
4%
16%
27%
Source Build
Test Build
Mozilla’s build coupling is very high!
.mk?.c
12%

74. Work Item Coupling is High
27
8%
20%
44%
4%
16%
27%
Source Build
Test Build
Mozilla’s build coupling is very high!
Very low due to
use of higher level
build abstraction
.mk?.c
12%

75. 28
People
How Tightly Coupled are
Build and Source Code?
.mk?.c
Size
9% of
all files
Evolution Coupling
Tightly
coupled
Build churn
>> source

76. 29
How Do Projects Distribute
Build Maintenance Work?
.mk?.c
Size
9% of
all files
Evolution Coupling
Tightly
coupled
People
Build churn
>> source

77. Identifying build maintainers
.c .c .c .ac.c
Source
Build
.c .c

78. Identifying build maintainers
Change
list #1
.c .c .c
Change
list #3
Change
list #4
.ac.c
Change
list #2
Source
Build
.c .c

79. Identifying build maintainers
Change
list #1
.c .c .c
Change
list #3
Change
list #4
.ac.c
Change
list #2
25% build
Source
Build
.c .c

80. Identifying build maintainers
Change
list #1
.c .c .c
Change
list #3
Change
list #4
.ac.c
Change
list #2
25% build
Source
Build
.c .c

81. Identifying build maintainers
Change
list #1
.c .c .c
Change
list #3
Change
list #4
.ac.c
Change
list #2
0% build
25% build
Source
Build
.c .c

82. Identifying build maintainers
Change
list #1
.c .c .c
Change
list #3
Change
list #4
.ac.c
Change
list #2
0% build
25% build
Source
Build
.c .c

83. Identifying build maintainers
Change
list #1
.c .c .c
Change
list #3
Change
list #4
.ac.c
Change
list #2
0% build 50% build
25% build
Source
Build
.c .c

84. Identifying build maintainers
Change
list #1
.c .c .c
Change
list #3
Change
list #4
.ac.c
Change
list #2
0% build 50% build
Source
Build
.c .c
≥

85. Identifying build maintainers
Change
list #1
.c .c .c
Change
list #3
Change
list #4
.ac.c
Change
list #2
0% build 50% build
Source
Build
.c .c
≥

86. If you maintain source code,
do you maintain build code?
31
25%22%
79%
% of developers

87. If you maintain source code,
do you maintain build code?
31
25%22%
79%
% of developers

88. Dispersed
Ownership:
Build maintenance is
distributed amongst
most team members
32

89. Dispersed
Ownership:
Build maintenance is
distributed amongst
most team members
32
Concentrated
Ownership:
A small team is
responsible for most
of the build maintenance

90. 33
How Do Projects Distribute
Build Maintenance Work?
.mk?.c
Size
9% of
all files
Evolution Coupling
Tightly
coupled
People
Dispersed or
concentrated
Build churn
>> source

91. Tool support needed
for build maintenance!
34
Build churn >>
source churn
Build maintenance required
for 27% of source code tasks

92. Threats toValidity Studied
projects
All
projects
35
Semi-automatic
classification

93. 36

94. 36
PLplot

95. 37
PLplot
Size
9% of
all files

96. 38
PLplot
Size
9% of
all files
Evolution
Build churn
>> source

97. 39
PLplot
.mk?.c
Size
9% of
all files
Evolution Coupling
Tightly
coupled
Build churn
>> source

98. 40
PLplot
.mk?.c
Size
9% of
all files
Evolution Coupling
Tightly
coupled
People
Dispersed or
concentrated
Build churn
>> source

99. 41
PLplot
.mk?.c
Size
9% of
all files
Evolution Coupling
Tightly
coupled
People
Dispersed or
concentrated
Build churn
>> source

100. 41
PLplot
.mk?.c
Size
9% of
all files
Evolution Coupling
Tightly
coupled
People
Dispersed or
concentrated
Questions?
Build churn
>> source