Download as PDF, PPTX
![THE INFLUENCE OF ORGANIZATIONAL STRUCTURE ON
SOFTWARE QUALITY: AN EMPIRICAL CASE STUDY
Nachiappan Nagappan
Microsoft Research
Redmond, WA, USA
nachin@microsoft.com
Brendan Murphy
Microsoft Research
Cambridge, UK
bmurphy@microsoft.com
Victor R. Basili
University of Maryland
College Park, MD, USA
basili@cs.umd.edu
ABSTRACT
Often software systems are developed by organizations consisting
of many teams of individuals working together. Brooks states in
the Mythical Man Month book that product quality is strongly
affected by organization structure. Unfortunately there has been
little empirical evidence to date to substantiate this assertion. In
this paper we present a metric scheme to quantify organizational
complexity, in relation to the product development process to
identify if the metrics impact failure-proneness. In our case study,
the organizational metrics when applied to data from Windows
Vista were statistically significant predictors of failure-proneness.
The precision and recall measures for identifying failure-prone
binaries, using the organizational metrics, was significantly higher
than using traditional metrics like churn, complexity, coverage,
dependencies, and pre-release bug measures that have been used
to date to predict failure-proneness. Our results provide empirical
evidence that the organizational metrics are related to, and are
effective predictors of failure-proneness.
Categories and Subject Descriptors
D.2.8 [Software Engineering]: Software Metrics – complexity
measures, performance measures, process metrics, product
metrics.
General Terms
Measurement, Reliability, Human Factors.
Keywords
Organizational structure, Failures, Code churn, People, Empirical
Studies.
1. INTRODUCTION
Software engineering is a complex engineering activity. It
involves interactions between people, processes, and tools to
develop a complete product. In practice, commercial software
development is performed by teams consisting of a number of
individuals ranging from the tens to the thousands. Often these
people work via an organizational structure reporting to a manager
or set of managers.
The intersection of people [9], processes [29] and organization
[33] and the area of identifying problem prone components early
in the development process using software metrics (e.g. [13, 24,
28, 30]) has been studied extensively in recent years. Early
indicators of software quality are beneficial for software engineers
and managers in determining the reliability of the system,
estimating and prioritizing work items, focusing on areas that
require more testing, inspections and in general identifying
“problem-spots” to manage for unanticipated situations. Often
such estimates are obtained from measures like code churn, code
complexity, code coverage, code dependencies, etc. But these
studies often ignore one of the most influential factors in software
development, specifically “people and organizational structure”.
This interesting fact serves as our main motivation to understand
the intersection between organizational structure and software
quality: How does organizational complexity influence quality?
Can we identify measures of the organizational structure? How
well do they do at predicting quality, e.g., do they do a better job
of identifying problem components than earlier used metrics?
Conway’s Law states that “organizations that design systems are
constrained to produce systems which are copies of the
communication structures of these organizations.” [8]. Similarly,
Fred Brooks argues in the Mythical Man Month [6] that the
product quality is strongly affected by org structure. With the
advent of global software development where teams are
distributed across the world the impact of organization structure
on Conway’s law [15] and its implications on quality is
significant. To the best of our knowledge there has been little or
no empirical evidence regarding the relationship/association
between organizational structure and direct measures of software
quality like failures.
In this paper we investigate this relationship between
organizational structure and software quality by proposing a set of
eight measures that quantify organizational complexity. These
eight measures provide a balanced view of organizational
complexity from the code viewpoint. For the organizational
metrics, we try to capture issues such as organizational distance of](https://image.slidesharecdn.com/conwayslawrevisited-151130085231-lva1-app6891/85/ConwaysLawRevisited-35-320.jpg)
Uploaded byJens Himmelreich
ConwaysLawRevisited
Der Artikel untersucht, wie das Design von Systemen durch die Kommunikationsstrukturen der Organisationen beeinflusst wird, die sie entwerfen, und beschreibt die Phasen des Designprozesses. Conway's Gesetz besagt, dass Organisationen Systeme schaffen, die deren Kommunikationsstrukturen widerspiegeln, was die Flexibilität und Effektivität des Designs beeinträchtigen kann. Eine empirische Studie zeigt, dass die Organisationsstruktur signifikante Vorhersagen über die Fehleranfälligkeit von Software ermöglicht und somit die Qualität beeinflusst.
More Related Content
![Lect 1 Number systems and base conversions. [Autosaved].pptx](https://cdn.slidesharecdn.com/ss_thumbnails/lect1numbersystemsandbaseconversions-260111134109-67c2d865-thumbnail.jpg?width=640&height=640&fit=bounds)
ConwaysLawRevisited
- 1. SOZIALE SYSTEME MICROSERVICES AGILITÄT CONWAYS LAWREVISITEDJens Himmelreich - XPDays 2015
- 2.
- 3. HOW DO COMMITTEES INVENT?by MELVINE. CONWAY That kind of intellectual activity which creates a useful whole from its diverse parts may be called the design of a system. Whether the particular activity is the creation of specifica- tions for a major weapon system, the formation of a rec- ommendation to meet a social challenge, or the program- ming of a computer, the general activity is largely the same. Typically, the objective of a design organization is the creation and assembly of a document containing a coherent- ly structured body of information. We may name this information the system design. It is typically produced for a sponsor who usually desires to carry out some activity guided by the system design. For example, a public official may wish to propose legislation to avert a recurrence of a recent disaster, so he appoints a team to explain the catas- trophe. Or a manufacturer needs a new product and desig- nates a product planning activity to specify what should be introduced. The design organization may or may not be involved in the construction of the system it designs. Frequently, in public affairs, there are policies which discourage a group's acting upon its own recommendations, whereas, in private industry, quite the opposite situation often prevails. It seems reasonable to suppose that the knowledge that one will have to carry out one's own recommendations or that this task will fall to others, probably affects some design choices which the individual designer is cailed upon to make. Most design activity requires continually making choices. Many of these choices may be more than design decisions; they may also be personal decisions the designer makes about his own future. As we shall see later, the incentives which exist in a conventional management en- vironment can motivate choices which subvert the intent of the sponsor.! stages of design The initial stages .of a design effort are concerned more with structuring of the design activity than with the system itsel£.2 The full-blown design activity cannot proceed until certain preliminary milestones are passed. These include: 1. Understanding of the boun9aries, both on the design activity and on the system to be designed, placed by the sponsor and by the worltl's realities. 2. Achievement of a preliminary notion of the system's organization so that design task groups can be mean- ingfully assigned. We shall see in detail later that the very act of organiz- 1 A related, but much more comprehensive discussion of the behavior of system-designing organizations is found in John Kenneth Galbraith's, The New Industrial State (Boston, Houghton Mifflin, 1967). See especially Chapter VI, "The Technostructur<!." 2 For o discussion of the problems which may arise when the design activity takes the form of o project in a functional environment, see C. J. Middleton, "How to Set Up o Project Organization," Harvard Business Review, March-April, 1967, p. 73. 28 design organization criteria ing a design team means that certain design decisions have already been made, explicitly or otherwise. Given any design team organization, there is a class of design alterna- tives which cannot be effectively pursued by such an organization because the necessary communication paths do not exist. Therefore, there is no such thing as a design group which is both organized and unbiased. Once the organization of the design team is chosen, it is possible to delegate activities to the subgroups of the organization. Every time a delegation is made and some- body's scope of inquiry is narrowed, the class of design alternatives which can be effectively pursued is also nar- rowed. Once scopes of activity are defined, a coordination prob- lem is created. Coordination among task groups, although it appears to lower the productivity of the individual in the small group, provides the only possibility that the separate task groups will be able to consolidate their efforts into a unified system design. Thus the life cycle of a system design effort proceeds through the following general stages: 1. Drawing of boundaries according to the ground rules. 2. Choice of a preliminary system concept. 3. Organization of the design activity and delegation of tasks according to that concept. 4. Coordination among delegated tasks. 5. Consolidation of subdesigns into a single design. It is possible that a given design activity will not pro- ceed straight through this list. It might conceivably reorga- nize upon discovery of a new, and obviously superior, design concept; but such an appearance of uncertainty is unflattering, and the very act of voluntarily abandoning a creation is painful and expensive. Of course, from the Dr. Conway is manager, pe- ripheral systems research, at Sperry Rand's Univac Div., where he is working on recog- nition of continuous speech. He has previously been a research associate at Case Western Re- serve Univ., and a software consultant. .He has an MS in physics from CaiTech and a PhD in math from Case. :C»ATAMATION
- 4. HOW DO COMMITTEES INVENT?Melvin Conway- Datamation, April 1968
- 5. ORGANIZATIONS WHICH DESIGN SYSTEMSARE CONSTRAINED TO PRODUCE DESIGNS WHICH ARE COPIES OF THE COMMUNICATION STRUCTURES OF THESE ORGANIZATIONS. MELVIN CONWAY
- 6. IF YOU HAVEFOUR GROUPS WORKING ON A COMPILER, YOU'LL GET A 4-PASS COMPILER. HACKERDICTIONARY
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17. ORGANISATIONist eine Hypothesefür die DOMÄNE
- 18.
- 19.
- 20. GIVEN ANY DESIGNTEAM ORGANIZATION, THERE IS A CLASS OF DESIGN ALTERNATIVES WHICH CANNOT BE EFFECTIVLEY PURSUED BY SUCH AN ORGANIZATION … MELVIN CONWAY
- 21. BECAUSE THE DESIGNWHICH OCCURS FIRST IS ALMOST NEVER THE BEST POSSIBLE … THEREFORE, FLEXIBILITY OF ORGANIZATION IS IMPORTANT TO EFFECTIVE DESIGN. MELVIN CONWAY
- 22.
- 23.
- 24. Copyright © 2007,2008, 2011 by Alan MacCormack, John Rusnak, and Carliss Baldwin Working papers are in draft form. This working paper is distributed for purposes of comment and discussion only. It may not be reproduced without permission of the copyright holder. Copies of working papers are available from the author. Exploring the Duality between Product and Organizational Architectures: A Test of the “Mirroring” Hypothesis Alan MacCormack John Rusnak Carliss Baldwin Working Paper 08-039
- 25. EXPLORING THE DUALITYBETWEEN PRODUCT AND ORGANIZATIONAL ARCHITECTURES: A TEST OF THE MIRRORING HYPOTHESISAlan McCormick, John Rusnak, Carliss Baldwin Harvard Business School Working Paper, 27 Mar 2008
- 26. MIRRORING HYPOTHESES „Loosely-coupled organizations willtend to develop products with more modular architecture than tightly-coupled organizations.“
- 27. MIRRORING HYPOTHESES „Loosely-coupled organizations willtend to develop products with more modular architecture than tightly-coupled organizations.“
- 28. MIRRORING HYPOTHESES „Loosely-coupled organizations willtend to develop products with more modular architecture than tightly-coupled organizations.“
- 29. MIRRORING HYPOTHESES „Loosely-coupled organizations willtend to develop products with more modular architecture than tightly-coupled organizations.“
- 30. MIRRORING HYPOTHESES „Loosely-coupled organizations willtend to develop products with more modular architecture than tightly-coupled organizations.“
- 31.
- 32. GNUCASH MYBOOKS ABIWORD STARWRITER GNUMERIC STARCALC LINUX 2.1.32 SOLARIS LINUX 2.6.8 XNU MYSQL BERKELEYDB 7,74% 47,14% 8,25% 41,77% 23,62% 54,31% 7,18% 22,59% 7,21% 24,83% 11,3% 43,23%
- 33. IN ALL PAIRSWE EXAMINE, THE LOOSELY-COUPLED ORGANIZATION DEVELOPS A PRODUCT WITH A MORE MODULAR DESIGN THAN THAT OF THE TIGHTLY-COUPLED ORGANIZATION. MACCORMACK, RUSNACK, BALDWIN
- 34.
- 35. THE INFLUENCE OFORGANIZATIONAL STRUCTURE ON SOFTWARE QUALITY: AN EMPIRICAL CASE STUDY Nachiappan Nagappan Microsoft Research Redmond, WA, USA nachin@microsoft.com Brendan Murphy Microsoft Research Cambridge, UK bmurphy@microsoft.com Victor R. Basili University of Maryland College Park, MD, USA basili@cs.umd.edu ABSTRACT Often software systems are developed by organizations consisting of many teams of individuals working together. Brooks states in the Mythical Man Month book that product quality is strongly affected by organization structure. Unfortunately there has been little empirical evidence to date to substantiate this assertion. In this paper we present a metric scheme to quantify organizational complexity, in relation to the product development process to identify if the metrics impact failure-proneness. In our case study, the organizational metrics when applied to data from Windows Vista were statistically significant predictors of failure-proneness. The precision and recall measures for identifying failure-prone binaries, using the organizational metrics, was significantly higher than using traditional metrics like churn, complexity, coverage, dependencies, and pre-release bug measures that have been used to date to predict failure-proneness. Our results provide empirical evidence that the organizational metrics are related to, and are effective predictors of failure-proneness. Categories and Subject Descriptors D.2.8 [Software Engineering]: Software Metrics – complexity measures, performance measures, process metrics, product metrics. General Terms Measurement, Reliability, Human Factors. Keywords Organizational structure, Failures, Code churn, People, Empirical Studies. 1. INTRODUCTION Software engineering is a complex engineering activity. It involves interactions between people, processes, and tools to develop a complete product. In practice, commercial software development is performed by teams consisting of a number of individuals ranging from the tens to the thousands. Often these people work via an organizational structure reporting to a manager or set of managers. The intersection of people [9], processes [29] and organization [33] and the area of identifying problem prone components early in the development process using software metrics (e.g. [13, 24, 28, 30]) has been studied extensively in recent years. Early indicators of software quality are beneficial for software engineers and managers in determining the reliability of the system, estimating and prioritizing work items, focusing on areas that require more testing, inspections and in general identifying “problem-spots” to manage for unanticipated situations. Often such estimates are obtained from measures like code churn, code complexity, code coverage, code dependencies, etc. But these studies often ignore one of the most influential factors in software development, specifically “people and organizational structure”. This interesting fact serves as our main motivation to understand the intersection between organizational structure and software quality: How does organizational complexity influence quality? Can we identify measures of the organizational structure? How well do they do at predicting quality, e.g., do they do a better job of identifying problem components than earlier used metrics? Conway’s Law states that “organizations that design systems are constrained to produce systems which are copies of the communication structures of these organizations.” [8]. Similarly, Fred Brooks argues in the Mythical Man Month [6] that the product quality is strongly affected by org structure. With the advent of global software development where teams are distributed across the world the impact of organization structure on Conway’s law [15] and its implications on quality is significant. To the best of our knowledge there has been little or no empirical evidence regarding the relationship/association between organizational structure and direct measures of software quality like failures. In this paper we investigate this relationship between organizational structure and software quality by proposing a set of eight measures that quantify organizational complexity. These eight measures provide a balanced view of organizational complexity from the code viewpoint. For the organizational metrics, we try to capture issues such as organizational distance of
- 36. THE INFLUENCE OFORGANISATIONAL STRUCTURE ON SOFTWARE QUALITY: AN EMPIRICAL CASE STUDYNachiappan Nagappan, Brendon Murphy, Victor R. Basili Microsoft Research TechReport, January 2008
- 37. IN THIS PAPERWE PRESENT A METRIC SCHEME TO QUANTIFY ORGANIZATIONAL COMPLEXITY, IN RELATION TO THE PRODUCT DEVELOPMENT PROCESS TO IDENTIFY IF THE METRICS IMPACT FAILURE-PRONENESS. NAGAPPAN, MURPHY, BASILI
- 38. ORGANIZATIONAL STRUCTURE CODE CHURN CODECOMPLEXITY DEPENDENCIES CODE COVERAGE PRE-RELEASE BUGS 86,2% 78,6% 79,3% 74,4% 83,3% 73,8% 84,0% 79,9% 66,0% 69,9% 54,4% 62,9% PRECISION RECALL
- 39. THE PRECISION ANDRECALL MEASURES FOR IDENTIFYING FAILURE-PRONE BINARIES, USING THE ORGANIZATIONAL METRICS, WAS SIGNIFICANTLY HIGHER THAN USING TRADITIONAL METRICS. NAGAPPAN, MURPHY, BASILI
- 40.
- 41.
- 42.
- 43.
- 44.
- 45.
- 46.
- 47.
- 48.
- 49.
- 50.
- 51.
- 52.
- 53.
- 54.
- 55.
- 56.
- 57.
- 58.
- 59.
- 60.
- 61.
- 62.
- 63.
- 64.
- 65.
- 66.
- 67.
- 68.
- 69.
- 70.
- 71. MUSTERtemporal - funktional- sozial
- 72. DWAWII SH FRONT OP RECO TR SEA NEWS SEOATTR BILD PROD
- 73. DWAWII SH FRONT OP RECO TR SEA NEWS SEOATTR BILD PROD FEATURE (BILD) FEATURE (PREIS)
- 74.
- 75.
- 76.
- 77. KUNDE PRODUKT BILD SUCHEWARENKORB NAVIGATION EMPFEHLUNG DETAILSEITE MEINKONTO REDUNDANZ
- 78.
- 79.
- 80.
- 81.
- 82.
- 83. SUCHEN ENTDECKEN BEWERTENKAUFEN STEUERN KUNDENORIENTIERUNG
- 84. SUCHEN ENTDECKEN BEWERTENKAUFEN STEUERN TIME-TO-MARKET
- 85. SUCHEN ENTDECKEN BEWERTENKAUFEN STEUERN
- 86. SUCHEN ENTDECKEN BEWERTENKAUFEN STEUERN
- 87. SUCHEN ENTDECKEN BEWERTENKAUFEN STEUERN DB DB DB DB DB HTML HTML HTML HTML HTML A B C A B C A B C A B C A B C
- 88. SUCHEN ENTDECKEN BEWERTENKAUFEN STEUERN DB DB DB DB DB HTML HTML HTML HTML HTML VARIANTE PRODUKT KNOTEN PRODUKT WARENKORB POSTITION ARTIKEL
- 89. SUCHEN ENTDECKEN BEWERTENKAUFEN STEUERN DB DB DB DB DB HTML HTML HTML HTML HTML A B C A B C A B C A B C A B C HTML DETAIL HTML NAVIGATION HTML EMPFEHLUNG HTML MINIWARENKORB HTML ANREDE STYLEGUIDE
- 90. SUCHEN ENTDECKEN BEWERTENKAUFEN STEUERN DB DB DB DB DB HTML HTML HTML HTML HTML A B C A B C A B C A B C A B C HTMLHTML HTML HTML HTML RWD APP INSTORE
- 91.
- 92. SUCHEN ENTDECKEN BEWERTENKAUFEN STEUERN FACHLICHE ISOLATION Bounded Context
- 93. SUCHEN ENTDECKEN BEWERTENKAUFEN STEUERN ORGANISATORISCHE ISOLATION Separates Scrumteam
- 94. SUCHEN ENTDECKEN BEWERTENKAUFEN STEUERN TECHNISCHE ISOLATION Keine synchrone Kopplung
- 95.
- 96. SUCHEN ENTDECKEN BEWERTENKAUFEN STEUERN REST-INTEGRATION smart endpoints and dump pipes
- 97. BEWERTEN PIM MICROSERVICES synchrone Kopplung erlaubt DETAIL SEITEBACKOFFICE PRODUKT VERGLEICH
- 98. SUCHEN ENTDECKEN BEWERTENKAUFEN STEUERN PAGER PAGER PAGER PAGER PAGER BETRIEBLICHE ISOLATION You Built it - you run it!
- 99.
- 100.
- 101.
- 102. ORGANISATION SYSTEM Wie spiegelt sichmeine Organisation im technischen System?
- 103. DOMÄNE ORGANISATION Welche Hypothese überdie Domäne stellt meine Organisation dar?
- 104.
- 105. QUELLEN HOW DO COMMITTEESINVENT? http://www.melconway.com/Home/pdf/committees.pdf EXPLORING THE DUALITY BETWEEN PRODUCT AND ORGANIZATIONAL ARCHITECTURES: A TEST OF THE MIRRORING HYPOTHESIS http://www.hbs.edu/research/pdf/08-039.pdf THE INFLUENCE OF ORGANISATIONAL STRUCTURE ON SOFTWARE QUALITY: AN EMPIRICAL CASE STUDY http://research.microsoft.com/pubs/70535/tr-2008-11.pdf The colors and fonts are inspired by Phil Hawksworth http://hawksworx.com/ THIS WORK IS LICENSED UNDER A CREATIVE COMMONS ATTRIBUTION-NONCOMMERCIAL-NODERIVS 3.0 UNPORTED LICENSE