This document proposes applying manufacturing performance figures to measure software development excellence. It discusses challenges in monitoring software development in manufacturing companies. The authors believe manufacturing KPIs can be applied to software since managers understand these KPIs. The document outlines related work transferring methods from manufacturing to software. It then proposes specific manufacturing KPIs that can be mapped to software metrics to measure quantity and quality. An industrial partner validated the approach technically by measuring prototype data and gained management support through interviews. While limited, the validation was positive. More work is needed to generalize the approach.

1. Andreas Deuter, Hans-Jürgen Koch IWSM-Mensura 2015
Applying manufacturing performance figures to
measure software development excellence

2. Andreas Deuter, Hans-Jürgen Koch IWSM-Mensura 2015
Ø Challenges
Ø Related Work
Ø Proposed approach
Ø Application of the approach
Ø Conclusion and future work
Content
2

3. Andreas Deuter, Hans-Jürgen Koch IWSM-Mensura 2015
Challenges
Ø Huge changes in traditional manufacturing driven by IT-
technology (Internet-of-Things, Cyber-physical systems)
Ø Products integrated in new flexible plants increase their
innovations by software
Ø Today, product manufacturing is one of the most important
business process in manufacturing companies
Ø It is monitored with manufacturing performance figures
ensuring manufacturing excellence
Ø New focus on manufacturing process as software is not
“assembled” in production lines
What is the problem?
3
Software development needs to be monitored in
manufacturing companies.

4. Andreas Deuter, Hans-Jürgen Koch IWSM-Mensura 2015
Challenges
Ø These examples of manufacturing performance figures are
perfectly understood by the management today.
Ø Therefore we think, that:
1. The manufacturing performance figures used today to monitor
manufacturing excellence can be applied to measure the software
development excellence.
2. The application of these well-understood performance figures creates
value by increased management attention towards software
development.
What do we think?
4

5. Andreas Deuter, Hans-Jürgen Koch IWSM-Mensura 2015
Related Work
Ø Agility extension of
traditional V-model
Ø Reduces Work in
Progress (WiP)
Ø Documents are
container of Work Items
Ø Links between Work
Items to form “V”
Sliced V-model [1]
5
[1] A. Deuter, “Slicing the V-model - Reduced effort, higher flexibility” in Proceedings of 8th International Conference on Global Software Engineering,
ICGSE’13, 2013, pp. 1–10.

6. Andreas Deuter, Hans-Jürgen Koch IWSM-Mensura 2015
Related Work
Ø Productivity increases in manufacturing motivates to
investigate applicable methods for software
Ø Prominent example is Kanban[5], which has been transferred
to software[6]
Ø Mixed picture, if manufacturing methods are applicable to
software – no’s[7] and yes’s[8]
Ø Research topic with little focus
Transfer from manufacturing to software
6
[5] Liker, J.: The Toyota Way. 1st edn. McGraw-Hill (2004)
[6] Hiranabe, K.: Kanban applied to software development: from agile to lean. http://www.infoq.com/articles/hiranabe-lean-agile-kanban (2008) [2015-04-24].
[7] Binder, R.: Can a manufacturing quality model work for software? Software, IEEE 14(5) (Sep 1997) 101
[8] Schneidewind, N.: What can software engineers learn from manufacturing to improve software process and product? In: Intelligent Information Management.
Volume 1. (2009)

7. Andreas Deuter, Hans-Jürgen Koch IWSM-Mensura 2015
Related Work
Ø Numerous articles, books and studies [2][3][4]
Ø Question to managers: “Do you understand the meaning of
these performance figures?“
• Answer: “Rather no”
Ø Option A:
• Explain managers software performance figures
Ø Option B:
• Use known manufacturing performance figures and apply them to
software
• We chose this option
Software Performance Figures
7
[2] Fenton, N., Pfleeger, S.L.: Software metrics (2nd ed.): a rigorous and practical approach. PWS Publishing Co., Boston, MA, USA (1997)
[3] Kasunic, M.: A data specifcation for software project performance measures: Results of a collaboration on performance measurement. Technical Report CMU/SEI-
2008-TR-012, Software Engineering Institute, Carnegie Mellon University, Pittsburgh,Pennsylvania (2008)
[4] Hartmann, D., Dymond, R.: Appropriate agile measurement: using metrics and diagnostics to deliver business value. In: Agile Conference, 2006. (July 2006) 6

8. Andreas Deuter, Hans-Jürgen Koch IWSM-Mensura 2015
Proposed Approach
Ø Software documentation
size Docr
• Number of Work Items
Software performance figures: Quantity and Quality
8
Ø Requirements churn[5] Chr
• Size of code changes [KB] to
implement requirement
Ø Internal defect churn Chdi
• Size of code changes to fix
internally reported defects
Ø External defect churn Chdx
• Size of code changes to fix
externally reported defects
[5] Sjoberg, D.I., Johnsen, A., Solberg, J.: Quantifying the effect of using Kanban versus scrum: A case study. IEEE Software 29 (2012) 47{53

9. Andreas Deuter, Hans-Jürgen Koch IWSM-Mensura 2015
Proposed Approach
Ø Following more figures defined in the paper:
• Number of defect founds in internal testing activities
• Number of defect report externally
• Total number of defects
• Development duration
• Development effort
• Productivity
• Velocity
• Defect Density
Software performance figures: all others
9

10. Andreas Deuter, Hans-Jürgen Koch IWSM-Mensura 2015
Proposed Approach
Ø First past yield:
Mapping software to manufacturing performance figures
10
Requirements
Churn
Internal Defect
Churn
External Defect
Churn
Allsourcecodechanges
correctdefective
Ø Complaints rate:
Ø More mapping in the paper

11. Andreas Deuter, Hans-Jürgen Koch IWSM-Mensura 2015
Evaluation of the approach
Ø Phoenix Contact supplier of electrical and electronic
components for industrial applications.
• >14,000 employees (2014), 51 owned sales companies and more than
30 sales partners worldwide.
• Software activities are carried out in three locations Bad Pyrmont,
Lemgo (Germany), Ann Arbor (USA).
Ø Sliced V-Model implemented
using Polarion ALM [6]
Ø Prototype implementing
our approach above concept
Who is the industrial partner?
11
[6] Polarion. http://www.polarion.com, 2015.

12. Andreas Deuter, Hans-Jürgen Koch IWSM-Mensura 2015
Evaluation of the approach
Ø Measuring the prototype data by the prototype
Verification: Does the approach work technically?
12
Software performance figures
Manufacturing performance figures

13. Andreas Deuter, Hans-Jürgen Koch IWSM-Mensura 2015
Ø Measuring a real software product (some data are disguised)
Evaluation of the approach
Validation: Does the approach support management?
13
Manufacturing performance figures
Ø Interview with five managers (low level software manager,
one quality manager, three higher level managers)
Ø Asking several questions, giving us positive feedback about
our approach

14. Andreas Deuter, Hans-Jürgen Koch IWSM-Mensura 2015
Conclusion
Ø Limited validation on one real project asking just five persons
• Our joined project is ongoing
Ø Approach limited to specific process model (Sliced V-model)
and tool chain (Polarion/Subversion)
• Ok for this environment, but effort needed to generalize the approach
Ø Some manufacturing data could not be mapped yet
Ø However, a first step in the right direction !
Much more to do…
14

15. Andreas Deuter, Hans-Jürgen Koch IWSM-Mensura 2015
Questions and Discussion
15