The document discusses the quality, cost, and governance of open source software, highlighting various managerial models and practices within open source projects. It examines how contributions and decision-making differ between community-driven and company-led projects, noting that open governance can lead to both higher quality and higher costs. The findings suggest that while open source development might not be cheaper, it offers unique advantages like enhanced community feedback and the potential for faster, cooperative technological advancement.

1. Quality, Cost, and Governance of Open Source Software Chiara Francalanci [email_address] Milano, 24 ottobre, 2008

2. What is Open Source? A licensing model which respects OSI definition Software with source code available A cooperative model to develop software that relies on communities Software developed by volunteers in their spare time Applications to which everybody can contribute A new marketing and business model Linux

3. Open Source refers to different models Development and managerial model Focus of the this talk Business model Ideological model Licensing and distribution model

4. Most people have in mind community Open Source projects… Preliminary evidence Java text editor Hosted on SourceForge (850k download per year) Variable 150 developers, all volunteers Everybody can contribute Commit privileges are accessible based on merit Decisions are made by informally discussing issues in forums and mailing lists – lazy consensus There are no deadlines and everybody voluntarily claims the “unassigned” issues he can tackle Mailing lists, forums, IRC channels People never meet in person EXAMPLE jEdit Description Communi- cation Managerial style Developers

5. … but some Open Source projects actually are a company… Preliminary evidence CRM platform SugarCRM is also a company (1M users) Variable 90% of the code is contributed by 11 SugarCRM employees The development processes are very similar to those of traditional companies Roadmaps and deadlines are defined by the management Discussions are fostered on forums and the opinion of the community is asked by means of online polls, but final decisions are always made by the management Most of the developers work in the same location and have regular meetings EXAMPLE SugarCRM Description Communi- cation Managerial style Developers

6. … and some communities are indirectly led by a company Preliminary evidence Office automation suite Community project (35M downloads) Variable 90% of the code is contributed by 100 core developers, who are Sun employees The managerial model is very structured and based on hierarchical governance bodies “ Managers” are chosen based on merit, but actually Sun controls the project as it provides fulltime committed developers who sum up to 90% of the total manpower There are shared roadmaps and task assignments Mailing lists, but most of Sun developers work in Hamburg and meet daily EXAMPLE OpenOffice Description Communi- cation Managerial style Developers

7. There is a continuum between open and close : we need a framework to position SW projects The framework is based on the continuous study of 75 major Open Source projects and face to face interviews with project managers and community managers It defines quantitative criteria and evaluation scales for a number of different dimensions It does not evaluate the “goodness” of a project Characteristics Identify the dimensions that characterize different managerial approaches Assess the openness of a project Goals

8. We have analyzed 75 major Open Source projects* Value Most famous and diffused Commercial and communities (SF, Apache, Tigris, Java) Variable Developers 26 80 on average 900 KLOC on average * Ricerca condotta in collaborazione con ing. Eugenio Capra Criteria Size Face to face interviews MySql SugarCRM OpenOffice Eclipse Mozilla JavaDB PostgreSQL OpenSolaris jEdit … Examples of projects

9. We identified four managerial dimensions (1/2) Leading questions What percentage of the code is contributed by hired developers? Dimension Project leadership Contribution Ranging from… … to Most of the code developed by employees or hired persons (e.g., MySql, Eclipse) All the code contributed on a voluntary basis (e.g., Tomcat, Drupal) Does the project have a formal organization? How are decisions made? Led by a company Hierarchical structure (e.g., MySql) Informal management “ Lazy consensus” decisions (e.g., DoJo Toolkit)

10. We identified four managerial dimensions (2/2) Leading questions How do people communicate and interact? Dimension Testing Working practices Ranging from… … to Regular meetings (e.g., SugarCRM, OpenOffice) Wide use of online tools (forums, IRC, e-mails) No physical meetings (e.g., MySql) How much of the testing does rely on the community? Is there a Quality Assurance (QA) plan? Most of the testing is performed by QA experts before releasing (e.g., OpenOffice) All the testing is done by the community (e.g., OpenSolaris)

11. The managerial framework applied: MySql MySql as a company controls the governance of the project and makes decisions Contribution: Project Leadership: Working Practices: Testing: EXAMPLE 99% of the code is developed by employees Developers are located in 26 countries and work from home Developers widely use IRC channels combined with e-mails and online shared task lists to keep track Functional and cross-platform tests are done internally Integration tests are left to the community Overall, more than 50% of testing is done by the community 1 2 4 3 Open Close

12. A provocative finding Some companies adopt Open Source strategies to pursue their corporate objectives Some Open Source communities are indirectly boosted and led by companies Open Source is not only the world of geeks who write code overnight and during week-ends only for personal satisfaction

13. A company is a company Commercial Open Source Projects Code developed by hired developers External committers Statistic data from our sample Why a company would adopt an Open Source strategy? It is a broad marketing platform It is beneficial to the image of the company They can release both open community editions free of charge, and closed enterprise editions for paying customers They can sell support, training, and professional services They can get very early feedback from the community, including testing However, they can keep control over the code and ensure quality by means of defined QA and testing processes ~92% ~0%

14. Companies support Open Source communities Community Open Source Projects ~50% ~35% Code developed by hired developers Projects that hold regular meetings Statistic data from our sample Why for-profit corporations would fund Open Source projects? It often accelerates the development of a technology by getting more people to contribute to the project Open components are more easily accepted by the community It is a good mechanism for cooperating with other companies and developing standards It allows them to pursue their objectives without stepping to the front line It is beneficial to the image of the company

15. Common beliefs on OS software More open governance mechanisms also reduce costs, since: Development is voluntary/unpaid Higher quality reduces development costs Costs are shared Rationale Belief OS has higher quality More open governance mechanisms are beneficial to quality, since: Lower pressure from deadlines Greater motivation of developers Virtual coordination among developers through code structure OS has lower costs

16. A few legitimate common concerns… Most of the costs are often accounted to a single company Distributed coordination is more challenging and cumbersome Less formal governance may not force quality control explicitly Most developers are paid

17. Consolidated knowledge on quality in closed source traditional software Explanation Quality involves a cost in the short term A higher-quality software involves lower maintenance costs over time Evidence Quality is an investment Quality decreases over time Each maintenance intervention decreases software quality, unless companies invest in quality Quality degradation causes software replacement or refactoring Deadlines and pressure from customers make companies favor a short-term view Companies do not invest in quality Software maintenance costs soar to the point of making replacement (or refactoring) economically convenient No evidence of the long-term benefits of quality

18. Consolidated knowledge on costs in closed source traditional software Explanation Objectives are clearly stated Management optimizes job allocation Decisions are centralized and faster Evidence Formal governance reduces costs Distributed and informal coordination favor creativity and innovation but are more costly Face-to-face meetings are rare, therefore self-training and problem solving are slower Decisions based on “consensus” are slower Less formal roadmaps, deadlines, and schedules may translate into lower efficiency due to a lack of planning

19. Research questions Is the quality of Open Source higher than that of proprietary software? Does Open Source cost more or less than proprietary software? What are the real advantages of Open Source?

20. Cost metric Development effort = (   N   T) /  M Average 11.2 35.0 Time spent by developers on OS projects Hours/ week Com-mercial Apache Java. net SF 16.2 10.8 8.5 Source : survey of 3.346 developers and administrators involved in 268 different OS projects  = percentage of time that each developer/ administrator spends on an application N = number of developers + number of administrators  T = time between the release of two subsequent application versions  M = number of new methods

21. Quality metrics Definition Number of distinct classes to which a given class is coupled (excl. inheritance relationships) Metric Standard code-based metrics measured with an ad-hoc tool Number of relations in pairwise sets of classes Weighted method complexity for a class Maximum depth of the inheritance graph of each class Number of immediate subclasses of a given class Average values Apache SF Java Average 4.5 3.3 4.3 4.3 0.011 0.018 0.017 0.013 9.2 13.6 8.6 10.4 0.6 0.3 0.9 0.5 0.4 0.3 0.3 0.3 Coupling Between Objects (CBO) Coupling Factor (COF) Weighted Methods per Class (WMC) Depth of Inheritance Tree (DIT) Number Of Children (NOC)

22. The continuum between open and closed source projects along the governance dimension Working practices: degree to which the working and communication practices of a project are geographically distributed and virtual Code: percentage of code that is available under an OS license Project Leadership: degree to which the leadership of a project is hierarchical Contribution: amount of voluntary code development 4 1 2 3 Open Close

23. Correlation model* Not significant Significant Significant * Ricerca condotta in collaborazione con ing. Eugenio Capra e Francesco Merlo DESIGN QUALITY CBO COF NOC WMC DIT GOVERNANCE Code Contribution Leadership Working practices COST Results A more open governance is correlated with higher quality A more open governance is correlated with higher costs Quality is not correlated with costs

24. Observations Refactoring is continuous in OS projects (40% of new versions have higher quality with respect to the previous version) The long-term balance of quality investments is neither positive nor negative: quality is a zero-sum game Quality is necessary to operate with an open approach Quality metrics cannot be taken as predictors of maintenance effort in OS projects Quality does not translate into savings Effort is partly due to the coordination overhead

25. Why should companies adopt the OS development model? Quality is higher and visible to the community Feedback from the community is faster, richer, and more constructive (all types of feedback, on code, requirements, bugs, documentation) Contribution is from a broader community: faster and cheaper internationalization through business networking, especially for system software New marketing medium through word of mouth within communities OS image can help adoption, especially in industries where regulation promotes OS as a standard … OS is not cheaper

26. Why should companies adopt OS software? Design quality is higher The community helps select the best software solutions The community supports users through forums, mailing lists, wikis, etc. OS promotes interoperable standards that can help cooperation, especially across companies Depending on the governance model, users can play a more active role in the definition of the requirements for new functionalities Depending on the governance model, TCO can be lower