• Like

Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Icse2011 build maintenance

  • 257 views
Uploaded on

Slides for An Empirical Study of Build Maintenance Effort.

Slides for An Empirical Study of Build Maintenance Effort.

More in: Technology , Sports
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
257
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
20
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Cars have features like AC and Power windows...\nSoftware has features... Linux -> CPU freq scaling, pluggable module support\nYou need tools to build a car, and you also need tools...compilers and linkers...\n
  • Cars have features like AC and Power windows...\nSoftware has features... Linux -> CPU freq scaling, pluggable module support\nYou need tools to build a car, and you also need tools...compilers and linkers...\n
  • Cars have features like AC and Power windows...\nSoftware has features... Linux -> CPU freq scaling, pluggable module support\nYou need tools to build a car, and you also need tools...compilers and linkers...\n
  • Cars have features like AC and Power windows...\nSoftware has features... Linux -> CPU freq scaling, pluggable module support\nYou need tools to build a car, and you also need tools...compilers and linkers...\n
  • Cars have features like AC and Power windows...\nSoftware has features... Linux -> CPU freq scaling, pluggable module support\nYou need tools to build a car, and you also need tools...compilers and linkers...\n
  • Cars have features like AC and Power windows...\nSoftware has features... Linux -> CPU freq scaling, pluggable module support\nYou need tools to build a car, and you also need tools...compilers and linkers...\n
  • Cars have features like AC and Power windows...\nSoftware has features... Linux -> CPU freq scaling, pluggable module support\nYou need tools to build a car, and you also need tools...compilers and linkers...\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • devs: constantly have to rebuilt artifacts to test changes... bld sys incrementally updates builds\n\n\n
  • devs: constantly have to rebuilt artifacts to test changes... bld sys incrementally updates builds\n\n\n
  • devs: constantly have to rebuilt artifacts to test changes... bld sys incrementally updates builds\n\n\n
  • devs: constantly have to rebuilt artifacts to test changes... bld sys incrementally updates builds\n\n\n
  • devs: constantly have to rebuilt artifacts to test changes... bld sys incrementally updates builds\n\n\n
  • devs: constantly have to rebuilt artifacts to test changes... bld sys incrementally updates builds\n\n\n
  • devs: constantly have to rebuilt artifacts to test changes... bld sys incrementally updates builds\n\n\n
  • devs: constantly have to rebuilt artifacts to test changes... bld sys incrementally updates builds\n\n\n
  • devs: constantly have to rebuilt artifacts to test changes... bld sys incrementally updates builds\n\n\n
  • devs: constantly have to rebuilt artifacts to test changes... bld sys incrementally updates builds\n\n\n
  • devs: constantly have to rebuilt artifacts to test changes... bld sys incrementally updates builds\n\n\n
  • devs: constantly have to rebuilt artifacts to test changes... bld sys incrementally updates builds\n\n\n
  • devs: constantly have to rebuilt artifacts to test changes... bld sys incrementally updates builds\n\n\n
  • Firefox 3.0 was built and delivered incorrectly\nusers in a networked environment address/search bar broken\ndue to an incorrect version of the SQLite library being linked in build process\nFix was delivered 4 months late in a service pack (3.0.1)\n
  • Firefox 3.0 was built and delivered incorrectly\nusers in a networked environment address/search bar broken\ndue to an incorrect version of the SQLite library being linked in build process\nFix was delivered 4 months late in a service pack (3.0.1)\n
  • Firefox 3.0 was built and delivered incorrectly\nusers in a networked environment address/search bar broken\ndue to an incorrect version of the SQLite library being linked in build process\nFix was delivered 4 months late in a service pack (3.0.1)\n
  • Firefox 3.0 was built and delivered incorrectly\nusers in a networked environment address/search bar broken\ndue to an incorrect version of the SQLite library being linked in build process\nFix was delivered 4 months late in a service pack (3.0.1)\n
  • Firefox 3.0 was built and delivered incorrectly\nusers in a networked environment address/search bar broken\ndue to an incorrect version of the SQLite library being linked in build process\nFix was delivered 4 months late in a service pack (3.0.1)\n
  • Firefox 3.0 was built and delivered incorrectly\nusers in a networked environment address/search bar broken\ndue to an incorrect version of the SQLite library being linked in build process\nFix was delivered 4 months late in a service pack (3.0.1)\n
  • Firefox 3.0 was built and delivered incorrectly\nusers in a networked environment address/search bar broken\ndue to an incorrect version of the SQLite library being linked in build process\nFix was delivered 4 months late in a service pack (3.0.1)\n
  • Firefox 3.0 was built and delivered incorrectly\nusers in a networked environment address/search bar broken\ndue to an incorrect version of the SQLite library being linked in build process\nFix was delivered 4 months late in a service pack (3.0.1)\n
  • Firefox 3.0 was built and delivered incorrectly\nusers in a networked environment address/search bar broken\ndue to an incorrect version of the SQLite library being linked in build process\nFix was delivered 4 months late in a service pack (3.0.1)\n
  • Based on survey results...\n
  • \n
  • Studied 10 projects of different:\n- prog language (c, c++, java)\n- domain (web communications, compilers, RDBMS, IDE...)\n- dev methodology (open, proprietary)\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • We find that the build system accounts for 9% of the build files in a project (median).\n- Small!\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • 1) churn of the build system is close to that of the source code (normalized)\n2) Prior research => High churn may produce bugs!\n
  • 1) churn of the build system is close to that of the source code (normalized)\n2) Prior research => High churn may produce bugs!\n
  • 1) churn of the build system is close to that of the source code (normalized)\n2) Prior research => High churn may produce bugs!\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • - measures strength of implication...\n
  • - measures strength of implication...\n
  • - measures strength of implication...\n
  • - measures strength of implication...\n
  • - measures strength of implication...\n
  • - measures strength of implication...\n
  • - measures strength of implication...\n
  • Counter intuitive and does not match with prior survey results (<12%)\n
  • - Mozilla is scary high\n- Eclipse-core and Jazz are considerably lower\n- Use of PDE build\n
  • - Mozilla is scary high\n- Eclipse-core and Jazz are considerably lower\n- Use of PDE build\n
  • - Mozilla is scary high\n- Eclipse-core and Jazz are considerably lower\n- Use of PDE build\n
  • - Mozilla is scary high\n- Eclipse-core and Jazz are considerably lower\n- Use of PDE build\n
  • \n
  • \n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - devs may not commit all changes in one revision\n
  • - Jazz src devs are often responsible for build dev\n- Git and Linux are less so\n
  • - Jazz distributes build work...\n
  • - Jazz distributes build work...\n
  • - Jazz distributes build work...\n
  • - Jazz distributes build work...\n
  • \n
  • Build maintenance is a nuisance for developers\nneed tools to help them to cope\ne.g., Jazz has tools, lowered build coupling to 4%\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Build maintenance is a nuisance for developers!\nneed tools to help them to cope!\n

Transcript

  • 1. An Empirical Study of Build Maintenance Effort Shane Bram Thanh H.D. Yasutaka Ahmed E.McIntosh Adams Nguyen Kamei Hassan
  • 2. 2
  • 3. 2
  • 4. 2
  • 5. The Build System Translates Sources into DeliverablesMake Autotools 2
  • 6. Step 1 - Configuration AutotoolsFeatures 3
  • 7. Step 1 - Configuration AutotoolsFeatures 3
  • 8. Step 1 - Configuration AutotoolsFeatures Tools 3
  • 9. Step 1 - Configuration AutotoolsFeatures Tools 3
  • 10. Step 2 - Construction Make Prescriptions 4
  • 11. Step 2 - Construction Make Prescriptions= 4
  • 12. Step 2 - Construction Make Prescriptions= 4
  • 13. Step 2 - Construction Make Prescriptions .c= 4
  • 14. Step 2 - Construction Make Prescriptions .c= 4
  • 15. Step 2 - Construction Make Prescriptions .c .o= 4
  • 16. Step 2 - Construction Make Prescriptions .c .o= Dependencies 4
  • 17. Step 2 - Construction Make Prescriptions .c .o= Dependencies 4
  • 18. Step 2 - Construction Make Prescriptions .c .o= Dependencies .o .o .o 4
  • 19. Step 2 - Construction Make Prescriptions .c .o= Dependencies .o .o .o .exe 4
  • 20. Build code is complex... 5
  • 21. Build Maintenance is aNuisance for Developers 6
  • 22. Build Maintenance is aNuisance for Developers 6
  • 23. Build Maintenance is a Nuisance for Developers MakeAutotools 6
  • 24. Build Maintenance is a Nuisance for Developers Make Migration to newer build technologyAutotools 6
  • 25. Build Bugs Even Affect the End User! 7
  • 26. Build Bugs Even Affect the End User!Address Bar 7
  • 27. Build Bugs Even Affect the End User!Address Bar Search Bar 7
  • 28. Build Bugs Even Affect the End User!Address Bar Search Bar 7
  • 29. Build Bugs Even Affect the End User!Address Bar Search Bar 7
  • 30. Build Systems Require 12% of a Developer’s Time (on average) Kumfert, G., and Epperly, T. Software in the DOE: The Hidden Overhead of the “Build”8
  • 31. Build Systems Require 12% of a Developer’s Time (on average) Build maintenanceslows development! Kumfert, G., and Epperly, T. Software in the DOE: The Hidden Overhead of the “Build” 8
  • 32. Four Dimensional Study of Build Maintenance Size Evolution Coupling People .c ? .mk 9
  • 33. Empirical Study of 10Systems (>25 MLOC) PLplot 10
  • 34. Empirical Study of 10Systems (>25 MLOC) PDE Build, ANT, GNU Autotools, Make PLplot 10
  • 35. Source, Test, or Build? .mk .c .c .ac .dat .h 11
  • 36. Source, Test, or Build? .c .c .mkSource Test Build .h .dat .acSource Test Build 11
  • 37. How Large is a Typical Build System?Size Evolution Coupling People .c ? .mk 12
  • 38. The Build Accounts for 9% of All Files (median)0.36 % Build files 31%0.270.18 16% 12% 12%0.09 10% 7% 8% 5% 3% 0 1% PLplot 13
  • 39. How Large is a Typical Build System? Size Evolution Coupling People .c ? .mk 9% ofall files 14
  • 40. How Does the Build Evolve? Size Evolution Coupling People .c ? .mk 9% ofall files 15
  • 41. File Churn Adjusted by System Size (# files) 3 source files a.c b.c x.c Jan 1 Feb 1 16
  • 42. File Churn Adjusted by System Size (# files) 3 source files a.c b.c x.c Jan 1 Feb 1 16
  • 43. File Churn Adjusted by System Size (# files) 3 source files a.c b.c x.c Jan 1 Feb 1 Source churn 2 ÷ 3 = 67% 16
  • 44. File Churn Adjusted by System Size (# files) 3 source files 2 Build specs a.c b.c x.c .mk cfg Jan 1 Feb 1 Source churn 2 ÷ 3 = 67% 16
  • 45. File Churn Adjusted by System Size (# files) 3 source files 2 Build specs a.c b.c x.c .mk cfg Jan 1 Feb 1 Source churn 2 ÷ 3 = 67% 16
  • 46. File Churn Adjusted by System Size (# files) 3 source files 2 Build specs a.c b.c x.c .mk cfg Jan 1 Feb 1 Source churn Build churn 2 ÷ 3 = 67% 1 ÷ 2 = 50% 16
  • 47. Build File Churn is on Par with the Source Churn 100(%) Source 80 Build 60File 40 20 0 PLplot 17
  • 48. Build File Churn is on Par with the Source Churn Build technology 100(%) migration Source 80 Build 60File 40 20 0 PLplot 17
  • 49. Line Churn Adjusted by System Size (# files) 3 source filesa.c b.c x.c Jan 1 Feb 1 18
  • 50. Line Churn Adjusted by System Size (# files) 3 source files 5 6 4 5a.c b.c x.c Jan 1 Feb 1 18
  • 51. Line Churn Adjusted by System Size (# files) 3 source files 5 6 4 5a.c b.c x.c Jan 1 Feb 1 Source Lines per Change list (5+6+4+5) ÷ 4 = 5 18
  • 52. Line Churn Adjusted by System Size (# files) 3 source files 5 6 4 5a.c b.c x.c Jan 1 Feb 1 Source Lines per Change list (5+6+4+5) ÷ 4 = 5 Adjust for system size 5÷3 = 1.67 18
  • 53. Build Line Churn is Higher than the Source ChurnAverage Lines per Change List (normalized by system size) 0.7 0.62 0.525 Source 0.52 Build 0.35 0.37 0.33 0.28 0.175 0.19 0 19
  • 54. How Does the Build Evolve? Size Evolution Coupling People .c ? .mk 9% of Build churnall files >> source 20
  • 55. How Tightly Coupled areBuild and Source Code? Size Evolution Coupling People .c ? .mk 9% of Build churnall files >> source 21
  • 56. .c ? .mk Changes, Change Lists, and Work ItemsChanges .c .c 22
  • 57. .c ? .mk Changes, Change Lists, and Work Items Changes .c .c Fix for bugChange Lists #1234 22
  • 58. .c ? .mk Changes, Change Lists, and Work Items Changes .c .c Fix for bugChange Lists #1234 22
  • 59. .c ? .mk Changes, Change Lists, and Work Items Changes .c .c .mk Fix for bugChange Lists #1234 22
  • 60. .c ? .mk Changes, Change Lists, and Work Items Changes .c .c .mk Fix for bug Missed code forChange Lists #1234 bug #1234 22
  • 61. .c ? .mk Changes, Change Lists, and Work Items Changes .c .c .mk Fix for bug Missed code forChange Lists #1234 bug #1234 22
  • 62. .c ? .mk Changes, Change Lists, and Work Items Changes .c .c .mk Fix for bug Missed code forChange Lists #1234 bug #1234Work items Bug #1234 22
  • 63. .c ? .mk Changes, Change Lists, and Work Items Changes .c .c .mk Fix for bug Missed code forChange Lists #1234 bug #1234Work items Bug #1234 22
  • 64. .c ? .mk Logical Coupling Example foo.cSource Files bar.c MakefileBuildFiles configure.ac CL1 CL2 CL3 CL4 CL5 Change Lists 23
  • 65. .c ? .mk Logical Coupling Example foo.cSource Files bar.c MakefileBuildFiles configure.ac CL1 CL2 CL3 CL4 CL5 Change Lists 23
  • 66. .c ? .mk Logical Coupling Example foo.cSource Files bar.c MakefileBuildFiles configure.ac CL1 CL2 CL3 CL4 CL5 Change Lists 23
  • 67. .c ? .mk Logical Coupling Example foo.cSource Files bar.c MakefileBuildFiles configure.ac CL1 CL2 CL3 CL4 CL5 Change Lists LC(Source Build) =2÷4 = 50% 23
  • 68. .c ? .mk Change List Coupling is Below 12% Estimate Source Build Test Build12% 7% 7% 7% 5% 5% 5% 4% 4% 4% 3% 3% 2% 24
  • 69. .c ? .mk Work Item Coupling is High 44% Source Build Test Build 27% 20% 16%12% 8% 4% 25
  • 70. .c ? .mk Work Item Coupling is High 44% Source Build Test Build 27% 20% 16%12% 8% 4% Mozilla’s build coupling is very high! 25
  • 71. .c ? .mk Work Item Coupling is High 44% Source Build Test Build Very low due to 27% use of higher level 20% build abstraction 16%12% 8% 4% Mozilla’s build coupling is very high! 25
  • 72. How Tightly Coupled areBuild and Source Code? Size Evolution Coupling People .c ? .mk 9% of Build churn Tightlyall files >> source coupled 26
  • 73. How Do Projects Distribute Build Maintenance Work? Size Evolution Coupling People .c ? .mk 9% of Build churn Tightly all files >> source coupled 27
  • 74. Identifying build maintainers Source Build.c .c .c .c .ac .c .c
  • 75. Identifying build maintainers Source Build.c .c .c .c .ac .c .cChange Change Change Changelist #1 list #2 list #3 list #4
  • 76. Identifying build maintainers Source Build.c .c .c .c .ac .c .cChange Change Change Change 25% buildlist #1 list #2 list #3 list #4
  • 77. Identifying build maintainers Source Build.c .c .c .c .ac .c .cChange Change Change Change 25% buildlist #1 list #2 list #3 list #4
  • 78. Identifying build maintainers Source Build.c .c .c .c .ac .c .cChange Change Change Change 25% buildlist #1 list #2 list #3 list #4 0% build
  • 79. Identifying build maintainers Source Build.c .c .c .c .ac .c .cChange Change Change Change 25% buildlist #1 list #2 list #3 list #4 0% build
  • 80. Identifying build maintainers Source Build.c .c .c .c .ac .c .cChange Change Change Change 25% buildlist #1 list #2 list #3 list #4 0% build 50% build
  • 81. Identifying build maintainers Source Build.c .c .c .c .ac .c .cChange Change Change Changelist #1 list #2 list #3 list #4 ≥ 0% build 50% build
  • 82. Identifying build maintainers Source Build.c .c .c .c .ac .c .cChange Change Change Changelist #1 list #2 list #3 list #4 ≥ 0% build 50% build
  • 83. If you maintain source code,do you maintain build code? % of developers 79% 22% 25% 29
  • 84. If you maintain source code,do you maintain build code? % of developers 79% 22% 25% 29
  • 85. Dispersed Ownership:Build maintenance isdistributed amongstmost team members 30
  • 86. Dispersed Ownership: Build maintenance is distributed amongst most team members Concentrated Ownership: A small team is responsible for mostof the build maintenance 30
  • 87. How Do Projects Distribute Build Maintenance Work? Size Evolution Coupling People .c ? .mk 9% of Build churn Tightly Dispersed or all files >> source coupled concentrated 31
  • 88. Tool support needed for build maintenance! Build maintenance required Build churn >>for 27% of source code tasks source churn 32
  • 89. Threats to Validity Studied projects All projects Semi-automatic classification 33
  • 90. 34
  • 91. PLplot34
  • 92. Size 9% ofall files PLplot 35
  • 93. Size Evolution 9% of Build churnall files >> source PLplot 36
  • 94. Size Evolution Coupling .c ? .mk 9% of Build churn Tightlyall files >> source coupled PLplot 37
  • 95. Size Evolution Coupling People .c ? .mk 9% of Build churn Tightly Dispersed orall files >> source coupled concentrated PLplot 38
  • 96. Size Evolution Coupling People .c ? .mk 9% of Build churn Tightly Dispersed orall files >> source coupled concentrated PLplot 39
  • 97. Size Evolution Coupling People .c ? .mk 9% of Build churn Tightly Dispersed orall files >> source coupled concentrated Questions? PLplot 39