Download as PDF, PPTX
![“...nothing can be
said to be certain,
except death and
taxes” - Benjamin Franklin
The Build “Tax”
An Empirical Study of Build
Maintenance Effort
S. McIntosh, B. Adams, T. H. D.
Nguyen, Y. Kamei, A. E. Hassan
[ICSE 2011]
Up to 27% of source
changes require build
changes, too!
5](https://image.slidesharecdn.com/crossprojectbuildcochange-150306104851-conversion-gate01/85/Cross-Project-Build-Co-change-Prediction-11-320.jpg)
![12
Prior work shows that within-project build
co-change prediction can be accurate
Mining Co-Change Information to
Understand when Build Changes
are Necessary
S. McIntosh, B. Adams, M.
Nagappan, A. E. Hassan
[ICSME 2014]
Build co-change
classifiers can achieve
an AUC of 0.60-0.88](https://image.slidesharecdn.com/crossprojectbuildcochange-150306104851-conversion-gate01/85/Cross-Project-Build-Co-change-Prediction-32-320.jpg)
Uploaded byShane McIntosh
Cross-Project Build Co-change Prediction
The document explores the role of build systems in software development, particularly focusing on the necessity of updating builds in response to source code changes, which can lead to build maintenance issues and breakages. It discusses the performance of build co-change prediction models, particularly emphasizing challenges related to small datasets, cross-project applicability, and the minority nature of build co-changes. The authors propose using transfer learning to enhance prediction accuracy and report improvements in model performance over baseline techniques in their experimental findings.
More Related Content
Similar to Cross-Project Build Co-change Prediction
![20260201 [FOSDEM] gomodjail - library sandboxing for Go modules.pdf](https://cdn.slidesharecdn.com/ss_thumbnails/20260201fosdemgomodjail-librarysandboxingforgomodules-260201225659-76609ec4-thumbnail.jpg?width=640&height=640&fit=bounds)
![谷歌留痕技术教程[ 𝙩𝙤𝙥 𝟮𝟯𝟯. 𝙘 𝙤𝙢 ]](https://cdn.slidesharecdn.com/ss_thumbnails/top233-260130173900-2eb784f9-thumbnail.jpg?width=640&height=640&fit=bounds)
Cross-Project Build Co-change Prediction
- 1. Cross-Project Build Co-change Prediction Shane McIntosh AhmedE. Hassan shanemcintosh@acm.org @shane_mcintosh shanemcintosh.org Emad Shihab David Lo Xin Xia
- 2. What is abuild system? Source code 2
- 3. What is abuild system? Source code Deliverable 2
- 4. .tex .c .cc .o .o .dvi .a .exe .pdf .deb Build systems describehow sources are translated into deliverables 3
- 5. The build systemis at the heart of techniques like Continuous Integration (CI) 4 .c .mk
- 6. The build systemis at the heart of techniques like Continuous Integration (CI) Commit 4 Commit 9719cf0 .c .mk
- 7. The build systemis at the heart of techniques like Continuous Integration (CI) Commit 4 Build Commit 9719cf0 .c .mk
- 8. The build systemis at the heart of techniques like Continuous Integration (CI) Commit 4 Build Test Commit 9719cf0 .c .mk
- 9. The build systemis at the heart of techniques like Continuous Integration (CI) Commit 4 Build Test Report Commit 9719cf0 wassuccessfullyintegrated Commit 9719cf0 .c .mk
- 10. The build systemis at the heart of techniques like Continuous Integration (CI) Commit 4 Build Test Report Commit 9719cf0 wassuccessfullyintegrated Commit 9719cf0 .c .mk
- 11. “...nothing can be saidto be certain, except death and taxes” - Benjamin Franklin The Build “Tax” An Empirical Study of Build Maintenance Effort S. McIntosh, B. Adams, T. H. D. Nguyen, Y. Kamei, A. E. Hassan [ICSE 2011] Up to 27% of source changes require build changes, too! 5
- 12. Neglected build maintenance isa frequent cause of build breakage 6 .c .mk
- 13. Neglected build maintenance isa frequent cause of build breakage Commit 6 Commit aedd38 .c .mk
- 14. Neglected build maintenance isa frequent cause of build breakage Commit 6 Commit aedd38 .c .mk
- 15. Neglected build maintenance isa frequent cause of build breakage Commit 6 Build Commit aedd38 .c .mk
- 16. Neglected build maintenance isa frequent cause of build breakage Commit 6 Build Test Commit aedd38 .c .mk
- 17. Neglected build maintenance isa frequent cause of build breakage Commit 6 Build Test Commit aedd38 .c .mk
- 18. Neglected build maintenance isa frequent cause of build breakage Commit 6 Build Test Report Commit aedd38 .c .mk Commit aedd38 broke the build!
- 19. Neglected build maintenance caneven impact end users 7
- 20. Neglected build maintenance caneven impact end users 7 Not working due to linking of incorrect SQLite library version
- 21. Neglected build maintenance caneven impact end users 7 Not working due to linking of incorrect SQLite library version When are build changes necessary?
- 22. 8 Overview of thestudied systems
- 23. 8 Overview of thestudied systems 29 years of historical data
- 24. 8 Overview of thestudied systems 29 years of historical data Proprietary and opensource systems
- 25. Grouping related changesaccording to the work items that they address 9
- 26. Grouping related changesaccording to the work items that they address .c .c .c Changes .mk 9
- 27. Missed code in #2121 Addfeature #2121 Fix for bug #1234 Grouping related changes according to the work items that they address .c .c .c Transactions Changes .mk 9
- 28. 2121 Missed code in #2121 Addfeature #2121 1234 Fix for bug #1234 Grouping related changes according to the work items that they address .c .c .c Transactions Work items Changes .mk 9
- 29. 1 2 .mk 10 We trainclassifiers to identify code changes that require build co-changes Work items .c.c .c Classification model Build change necessary No build change necessary
- 30. 1 2 .mk 10 We trainclassifiers to identify code changes that require build co-changes Work items .c .c .cClassification model Build change necessary No build change necessary
- 31. 1 2 .mk 11 Work items .c Build change necessary Nobuild change necessary Classification model We train classifiers to identify code changes that require build co-changes
- 32. 12 Prior work showsthat within-project build co-change prediction can be accurate Mining Co-Change Information to Understand when Build Changes are Necessary S. McIntosh, B. Adams, M. Nagappan, A. E. Hassan [ICSME 2014] Build co-change classifiers can achieve an AUC of 0.60-0.88
- 33. However, a largeamount of historical data was used to train the classifiers 13
- 34. However, a largeamount of historical data was used to train the classifiers 13
- 35. However, a largeamount of historical data was used to train the classifiers 13 What about new projects?
- 36. However, a largeamount of historical data was used to train the classifiers 13 What about new projects? …or projects withpoorly-recordedhistorical data?
- 37. However, a largeamount of historical data was used to train the classifiers 13 What about new projects? …or projects withpoorly-recordedhistorical data? Can we leverage these large corpora for the small ones?
- 38.
- 39. 14 How well dobuild co- change prediction models perform on sparse data? Precision Recall F1-score AUC 0 0.25 0.5 0.75 1 5% 50% 90%
- 40. 14 How well dobuild co- change prediction models perform on sparse data? Precision Recall F1-score AUC 0 0.25 0.5 0.75 1 5% 50% 90% Challenge 1: Very small datasets tend to yield models that under-perform
- 41. 14 How well dobuild co- change prediction models perform on sparse data? Precision Recall F1-score AUC 0 0.25 0.5 0.75 1 5% 50% 90% How well do build co- change prediction models perform on other datasets? Precision Recall F1-score AUC 0 0.25 0.5 0.75 1 Eclipse => Mozilla Jazz => Mozilla Lucene => Mozilla Challenge 1: Very small datasets tend to yield models that under-perform
- 42. 14 How well dobuild co- change prediction models perform on sparse data? Precision Recall F1-score AUC 0 0.25 0.5 0.75 1 5% 50% 90% How well do build co- change prediction models perform on other datasets? Precision Recall F1-score AUC 0 0.25 0.5 0.75 1 Eclipse => Mozilla Jazz => Mozilla Lucene => Mozilla Challenge 1: Very small datasets tend to yield models that under-perform Challenge 2: Cross-project build co- change models tend to under-perform
- 43. 15 Domain-specific project characteristicsmay limit the applicability of cross-project models Training corpus Testing corpus
- 44. Training corpus 16 Classification model Testing corpus Domain-specific project characteristicsmay limit the applicability of cross-project models
- 45. Training corpus 16 Classification model Testing corpus ? Domain-specific project characteristicsmay limit the applicability of cross-project models
- 46. 17 Using transfer learningto provide some domain knowledge to the training corpus Training corpus Testing corpus
- 47. Move some training datafrom target system to the training corpus 17 Using transfer learning to provide some domain knowledge to the training corpus Training corpus Testing corpus
- 48. 18 Training corpus Testing corpus Using transfer learningto provide some domain knowledge to the training corpus
- 49. 19 Training corpus Testing corpus Classification model Using transfer learningto provide some domain knowledge to the training corpus
- 50. 19 Training corpus Testing corpus Classification model ? Using transfer learningto provide some domain knowledge to the training corpus
- 51. 20 Challenge 3: Build co-changesare the minority
- 52. 20 Challenge 3: Build co-changesare the minority Only 8%-17% of changesare build co-changing
- 53. 21 Training corpus Testing corpus Use training corpusto find an appropriate threshold
- 54. 22 Training corpus Testing corpus Classification model Use training corpusto find an appropriate threshold
- 55. Set aside the testingcorpus 22 Training corpus Testing corpus Classification model Use training corpus to find an appropriate threshold
- 56. 23 Training corpus Classification model Use training corpusto find an appropriate threshold Training corpus
- 57. Incorrectly classified! 23 Training corpus Classification model Use training corpusto find an appropriate threshold Training corpus
- 58. 24 Use training corpusto find an appropriate threshold Training corpus Classification model
- 59. 24 Use training corpusto find an appropriate threshold Training corpus Classification model 1
- 60. 25 Use training corpusto find an appropriate threshold Training corpus Classification model Classification model 1 2
- 61. 25 Use training corpusto find an appropriate threshold Training corpus Classification model Classification model 1 2
- 62. 26 Use training corpusto find an appropriate threshold Classification model Classification model 1 2 … Classification model N
- 63. Ensemble of models usedon the testing corpus 26 Use training corpus to find an appropriate threshold Classification model Classification model 1 2 … Classification model N
- 64.
- 65. 27 Evaluating our approach Relative performance Trainingconfiguration sensitivity Source Target
- 66. 28 Evaluating our approach Relative performance Source Target Trainingconfiguration sensitivity
- 67. 29 Our approach outperformsbaseline cross-project approaches Eclipse Jazz Lucene Mozilla Average 0 0.25 0.5 0.75 1 Our approach Ordinary cross-project AdaBoost TrAdaBoost Worstmeasured F-score
- 68. 29 Our approach outperformsbaseline cross-project approaches Eclipse Jazz Lucene Mozilla Average 0 0.25 0.5 0.75 1 Our approach Ordinary cross-project AdaBoost TrAdaBoost Worstmeasured F-score 37%-42% improvement
- 69. 30 Our approach achievessimilar results to within-project models Eclipse Jazz Lucene Mozilla Average 0 0.25 0.5 0.75 1 Our approach Within-project Worstmeasured F-score
- 70. 30 Our approach achievessimilar results to within-project models Eclipse Jazz Lucene Mozilla Average 0 0.25 0.5 0.75 1 Our approach Within-project Only a 7% drop in performance Worstmeasured F-score
- 71. 31 Evaluating our approach Relative performance Source Target Trainingconfiguration sensitivity
- 72. 31 Evaluating our approach Relative performance 37%-42% improvement overbaseline Source Target Training configuration sensitivity
- 73. 31 Evaluating our approach Relative performance 37%-42% improvement overbaseline Only 7% drop of within-project F-measure Source Target Training configuration sensitivity
- 74. 32 Evaluating our approach Relative performance Source Target 37%-42% improvement overbaseline Only 7% drop of within-project F-measure Training configuration sensitivity
- 75. 33 Additional data fromthe target system slowly improves classifier performance Source Target F-score
- 76. 34 Evaluating our approach Relative performance Source Target 37%-42% improvement overbaseline Only 7% drop of within-project F-measure Training configuration sensitivity
- 77. 34 Evaluating our approach Relative performance Source Target 37%-42% improvement overbaseline Only 7% drop of within-project F-measure Training configuration sensitivity F-score tends to improve as more target system data becomes available
- 83.