An Empirical Study of Build System Migrations in Practice (ICSM 2012)


Published on

As the build system, i.e. the infrastructure that constructs executable deliverables out of source code and other resources, tries to catch up with the ever-evolving source code base, its size and already significant complexity keep on growing. Recently, this has forced some major software projects to migrate their build systems towards more powerful build system technologies. Since at all times software developers, testers and QA personnel rely on a functional build system to do their job, a build system migration is a risky and possibly costly undertaking, yet no methodology, nor best practices have been devised for it. In order to understand the build system migration process, we empirically studied two failed and two successful attempts of build system migration in two major open source projects, i.e. Linux and KDE, by mining source code repositories and tens of thousands of developer mailing list messages. The major contributions of this paper are: (a) isolating the phases of a common methodology for build system migrations, which is similar to the spiral model for source code development (multiple iterations of a waterfall process); (b) identifying four of the major challenges associated with this methodology: requirements gathering, communication issues, performance vs. complexity of build system code, and effective evaluation of build system prototypes; (c) detailed analysis of the first challenge, i.e., requirements gathering for the new build system, which revealed that the failed migrations did not gather requirements rigorously. Based on our findings, practitioners will be able to make more informed decisions about migrating their build system, potentially saving them time and money.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

An Empirical Study of Build System Migrations in Practice (ICSM 2012)

  1. 1. An Empirical Study ofBuild SystemMigrations in PracticeCase Studies on KDE and the Linux Kernel1R. Suvorov, B.Adams, M. Nagappan, A. E. Hassan,Y. Zou
  2. 2. Analogy: Car Assembly2+4x +assembly+
  3. 3. Build System Transforms Source Code& Resources into Executables3+bdSys+
  4. 4. bdSys is at the Heart ofthe Development Process!4bdSys+DevelopersQA Personnel Testers
  5. 5. bdSys Technology is “different”5
  6. 6. bdSys Technology is “different”5Our record so far is a project weinherited with an Ant scriptweighing in at 10,000 lines ofXML. Needless to say, this projectrequired an entire team devotedto keeping the build working - acomplete waste of resources.[Jez Humble & David Farley]KDE 4 is leaving the aging "autotool"build chain behind. Some developers,not only in KDE, like to nicknamethe autotools as "auto-hell"because of its difficult tocomprehend architecture.[Alexander Neundorf]Ive been examining the existingkernel configuration system, andconcluded that the best favor wecould do everybody involved withit is to take it out behind the barnand shoot it through thehead. [Eric Raymond]
  7. 7. ... yet a bdSys MustKeep on Evolving6MLOC
  8. 8. ..., in fact a bdSys ChangesRelatively More than Source Code71(c) McIntosh et al.:“An Empirical Study ofBuild Maintenance Effort”
  9. 9. ..., in fact a bdSys ChangesRelatively More than Source Code71ArgoUML Hibernate Eclipse GCC Git Linux Mozilla PLPlot PostgreSQL00.511.522.533.544.55# changes / source file# changes / build file(c) McIntosh et al.:“An Empirical Study ofBuild Maintenance Effort”
  10. 10. ..., in fact a bdSys ChangesRelatively More than Source Code71ArgoUML Hibernate Eclipse GCC Git Linux Mozilla PLPlot PostgreSQL00.511.522.533.544.55# changes / source file# changes / build file(c) McIntosh et al.:“An Empirical Study ofBuild Maintenance Effort”
  11. 11. ..., in fact a bdSys ChangesRelatively More than Source Code71ArgoUML Hibernate Eclipse GCC Git Linux Mozilla PLPlot PostgreSQL00.511.522.533.544.55# changes / source file# changes / build file(c) McIntosh et al.:“An Empirical Study ofBuild Maintenance Effort”
  12. 12. In Other Words ...8
  13. 13. In Other Words ...8Software Projects need HELPMaintaining their Build Systems
  14. 14. The “easy” way out: BuildSystem Migrations9
  15. 15. The “easy” way out: BuildSystem Migrations9
  16. 16. 10
  17. 17. 11
  18. 18. 12
  19. 19. 12Build system migrationsare not that easy! But why?
  20. 20. Major Contributions13CommonMigrationMethodologyMajorMigrationChallengesMajor Challenge:RequirementsGathering
  21. 21. Study Setup• Linux & KDE:• Two established projects: 21 & 16 years old• Large: 25 & 4.3 MLOC• 1 failed & 1 successful migration each• Data sources:• git repos: 300 K & 100 K commits• 4 mailing lists, over 1.5 million messages14
  22. 22. Quantitative Study ofDeveloper Commits• git repositories• Native git tools + bash scripts• Statistics on source & bdSys code:• Evolution• Churn• Distinguish between“normal” periods & migrations15
  23. 23. Qualitative Analysis ofDeveloper Communication• Personal emails to migration participants• Mailing lists• Mass-downloaded and analyzed• Statistics on number of messages:• By groups (roles)• By period of time• Distinguish between“normal” periods & migrations16
  24. 24. Major Contributions17CommonMigrationMethodologyMajorMigrationChallengesMajor Challenge:RequirementsGathering
  25. 25. 18CommonMigrationMethodology
  26. 26. The Spiral Model ofbdSys Migration191. Planning2. Risk Analysis3. Development4. Evaluation
  27. 27. Why? Migration isIncremental!• Linux kernel• 33K+ source code files• 2,300+ subdirectories• 5 major subsystems• KDE• 11K+ source code files• 22 packages20
  28. 28. 21CommonMigrationMethodology
  29. 29. 21CommonMigrationMethodologyspiral model!
  30. 30. 21CommonMigrationMethodologyspiral model!migrationhappensincrementally
  31. 31. 22CommonMigrationMethodologyMajorMigrationChallengesspiral model!migrationhappensincrementally
  32. 32. Communication AmongKey Participants23
  33. 33. Communication AmongKey Participants23bdSys Managermajor decisions
  34. 34. Communication AmongKey Participants23bdSys Managermajor decisionsbdSys Championleading bdSys dev’t
  35. 35. Communication AmongKey Participants23bdSys Managermajor decisionsbdSys Championleading bdSys dev’tbdSys ExpertbdSys contributor
  36. 36. Communication AmongKey Participants23bdSys Managermajor decisionsbdSys Championleading bdSys dev’tbdSys ExpertbdSys contributorCore Developersource contributor
  37. 37. Major Challenges• Communication issues• Build experts form a sub-community often reluctant tocommunicate effectively• Performance vs. complexity• Improving performance often means introducing hacks• Effective evaluation• When is migration "successfully finished"? How to measure?• Requirements gathering (see remainder)24
  38. 38. 25CommonMigrationMethodologyMajorMigrationChallengesspiral model!migrationhappensincrementally
  39. 39. 25CommonMigrationMethodologyMajorMigrationChallengesrequirementsspiral model!migrationhappensincrementally
  40. 40. 25CommonMigrationMethodologyMajorMigrationChallengesrequirementscommunicationspiral model!migrationhappensincrementally
  41. 41. 25CommonMigrationMethodologyMajorMigrationChallengesrequirementscommunicationperformancespiral model!migrationhappensincrementally
  42. 42. 25CommonMigrationMethodologyMajorMigrationChallengesrequirementscommunicationperformanceevaluationspiral model!migrationhappensincrementally
  43. 43. 26CommonMigrationMethodologyMajorMigrationChallengesMajor Challenge:RequirementsGatheringrequirementscommunicationperformanceevaluationspiral model!migrationhappensincrementally
  44. 44. Example bdSys Requirements• Platform independence• Two new major platforms: Mac OS X & Windows• High-level configuration• Single-pass methodology• Away with multiple-steps build!• Simple and platform-independent syntax27
  45. 45. Requirements GatheringProcess1. Stakeholder identification - who?• Who needs to be involved?2. Elicitation• What requirements are there?3. Analysis• Get all requirement data. Resolve conflicts, refine and organize.4. Specification & Maintenance• Document & order resulting requirements.28
  46. 46. How SCons failed• Stakeholder identification• Only 150 of 800 developers attended aKademy• Elicitation• Too little, too late: “trawled” for requirements• Analysis• Not enough attention paid to this phase!• Specification & Maintenance• Not done at all!29
  47. 47. 30“the main SCons tree has intolerable problems,and there are so many layers [...] that it barelyresembles what a normal SCons build system lookslike. Dont get me wrong; I use SCons at work and Ireally like it, but if it isnt capable of handling KDE thenwhy are we trying to make it do so? Is it reallyworth keeping a KDE specific build system (which iswhat we are moving towards)?[...] Perhaps it’s time to cut our losses and run.”[Jaison Lee]As a Result:
  48. 48. Rise and Fall of SCons31%max(LOC)
  49. 49. How succeeded• Stakeholder identification• Learned from SCons. Involved experts early.• Elicitation• Reuse some of SCons’ data.Active participation by experts!• Analysis• Identified conflicting requirements early.• Specification & Maintenance• An initiative by the build champion.32
  50. 50. 33CommonMigrationMethodologyMajorMigrationChallengesMajor Challenge:RequirementsGatheringspiral model!requirementscommunicationperformanceevaluationmigrationhappensincrementally
  51. 51. 33CommonMigrationMethodologyMajorMigrationChallengesMajor Challenge:RequirementsGatheringspiral model!requirementscommunicationperformanceevaluationmigrationhappensincrementallyinvolve allstakeholders
  52. 52. 33CommonMigrationMethodologyMajorMigrationChallengesMajor Challenge:RequirementsGatheringspiral model!requirementscommunicationperformanceevaluationmigrationhappensincrementallyinvolve allstakeholdersidentify &resolve conflicts
  53. 53. 33CommonMigrationMethodologyMajorMigrationChallengesMajor Challenge:RequirementsGatheringspiral model!requirementscommunicationperformanceevaluationmigrationhappensincrementallyinvolve allstakeholdersidentify &resolve conflictsefficient elicitation
  54. 54. 34
  55. 55. 34CommonMigrationMethodologyspiral model!migrationhappensincrementally
  56. 56. 34MajorMigrationChallengesrequirementscommunicationperformanceevaluationCommonMigrationMethodologyspiral model!migrationhappensincrementally
  57. 57. 34MajorMigrationChallengesrequirementscommunicationperformanceevaluationCommonMigrationMethodologyspiral model!migrationhappensincrementallyMajor Challenge:RequirementsGatheringinvolve allstakeholdersidentify &resolve conflictsefficient elicitation