This document summarizes the key findings of a study on build system migrations in large open source projects. It identifies common migration methodologies, challenges, and lessons learned. Major contributions include identifying a spiral model for incremental migrations. Key challenges are requirements gathering, communication between roles, and balancing performance vs complexity. The study compares a failed migration (SCons for KDE) to a successful one (autotools to CMake for Linux kernel) to understand best practices for requirements elicitation, analysis, and stakeholder involvement.
Measuring Program Comprehension: A Large-Scale Field Study with Professionals
Bdsys icsm v3.5
1. An Empirical Study of
Build System
Migrations in Practice
Case Studies on KDE and the Linux Kernel
1
R. Suvorov, B.Adams, M. Nagappan, A. E. Hassan,Y. Zou
6. bdSys Technology is “different”
5
Our record so far is a project weinherited with an Ant scriptweighing in at 10,000 lines ofXML. Needless to say, this projectrequired an entire team devotedto keeping the build working - acomplete waste of resources.[Jez Humble & David Farley]
KDE 4 is leaving the aging "autotool"
build chain behind. Some developers,
not only in KDE, like to nickname
the autotools as "auto-hell"
because of its difficult to
comprehend architecture.
[Alexander Neundorf]
I've been examining the existingkernel configuration system, andconcluded that the best favor wecould do everybody involved withit is to take it out behind the barnand shoot it through thehead. [Eric Raymond]
21. Study Setup
• Linux & KDE:
• Two established projects: 21 & 16 years old
• Large: 25 & 4.3 MLOC
• 1 failed & 1 successful migration each
• Data sources:
• git repos: 300 K & 100 K commits
• 4 mailing lists, over 1.5 million messages
14
22. Quantitative Study of
Developer Commits
• git repositories
• Native git tools + bash scripts
• Statistics on source & bdSys code:
• Evolution
• Churn
• Distinguish between
“normal” periods & migrations
15
23. Qualitative Analysis of
Developer Communication
• Personal emails to migration participants
• Mailing lists
• Mass-downloaded and analyzed
• Statistics on number of messages:
• By groups (roles)
• By period of time
• Distinguish between
“normal” periods & migrations
16
37. Major Challenges
• Communication issues
• Build experts form a sub-community often reluctant to
communicate effectively
• Performance vs. complexity
• Improving performance often means introducing hacks
• Effective evaluation
• When is migration "successfully finished"? How to measure?
• Requirements gathering (see remainder)
24
44. Example bdSys Requirements
• Platform independence
• Two new major platforms: Mac OS X & Windows
• High-level configuration
• Single-pass methodology
• Away with multiple-steps build!
• Simple and platform-independent syntax
27
45. Requirements Gathering
Process
1. Stakeholder identification - who?
• Who needs to be involved?
2. Elicitation
• What requirements are there?
3. Analysis
• Get all requirement data. Resolve conflicts, refine and organize.
4. Specification & Maintenance
• Document & order resulting requirements.
28
46. How SCons failed
• Stakeholder identification
• Only 150 of 800 developers attended aKademy
• Elicitation
• Too little, too late: “trawled” for requirements
• Analysis
• Not enough attention paid to this phase!
• Specification & Maintenance
• Not done at all!
29
47. 30
“the main SCons tree has intolerable problems,
and there are so many layers [...] that it barely
resembles what a normal SCons build system looks
like. Don't get me wrong; I use SCons at work and I
really like it, but if it isn't capable of handling KDE then
why are we trying to make it do so? Is it really
worth keeping a KDE specific build system (which is
what we are moving towards)?
[...] Perhaps it’s time to cut our losses and run.”
[Jaison Lee]
As a Result:
49. How succeeded
• Stakeholder identification
• Learned from SCons. Involved experts early.
• Elicitation
• Reuse some of SCons’ data.
Active participation by experts!
• Analysis
• Identified conflicting requirements early.
• Specification & Maintenance
• An initiative by the build champion.
32