The document summarizes a survey on decision making for component sourcing options in software engineering. It finds that companies typically consider developing components in-house as well as using commercial off-the-shelf (COTS) components, open source software (OSS), and outsourcing. Decisions are mainly based on expert judgment, with functionality being the most important criteria. Other important qualities include reliability, maintainability, performance, and security. Estimating component performance and reliability takes more time.

1. Supported byDeveloped by
CSO Decisions in Practice
- A Survey of Decision Making in Industry
Markus Borg | Industry Day, BTH | 2017-09-14
@mrksbrg
markus.borg@ri.se

2. Supported byDeveloped by
Make-or-buy decisions in software eng. are complex:
• Develop? Buy COTS? Outsource? OSS?
Decisions are mainly based on expert judgment
Functionality deal breaker, then reliability
Flickr:vladislav@munich

3. Supported byDeveloped by
Markus Borg
Development engineer, ABB, Malmö
• Editor and compiler development
• Process automation
PhD student, Lund University
• Machine learning for software engineering
• Bug reports and traceability
Senior researcher, RISE SICS AB, Lund
• Decision making
• V&V for self-driving cars

4. Supported byDeveloped by
Outline
Motivation
Method
Interaction! Results
DiscussionResults
Discussion

5. Supported byDeveloped by
Motivation
Component-based software engineering
• code reuse
• rapid development
Make-or-buy a component?
• Well-studied in manufacturing
• More difficult in R&D
Multiple sourcing options in software engineering
We study decision making in CSO selection

6. Supported byDeveloped by
Method
RQ1. Which CSOs are typically considered in
industry?
RQ2. What is the nature of the decision
process when selecting components?
RQ3. What component qualities are the most
important input to the decision process?

7. Supported byDeveloped by
Method

8. Supported byDeveloped by
353
Results
152
complete
353 answers in total
152 complete answers
201 partial answers
188 respondents >= Q9
36
>= Q9

9. Supported byDeveloped by
Demographics

10. Supported byDeveloped by
Repondent’s Role?

11. Supported byDeveloped by
Respondent’s Experience?
Experience
Education

12. Supported byDeveloped by
Primary Domain(s)?

13. Supported byDeveloped by
Organization’s Maturity?
Years on
market
Co-workers
in BU

14. Supported byDeveloped by
Organization’s Agility?

15. Supported byDeveloped by
RQ1:
Which CSOs?

16. Supported byDeveloped by
CSOs considered?
In-house
Outsource
COTS
OSS

17. Supported byDeveloped by
RQ1
Which CSOs are typically considered in industry?
 Confirm the existence of the phenomenon: CSO selection
 Most companies consider more than one CSO
 OSS and COTS are equally common
 Agile companies more likely to consider multiple CSOs
• More inclined to consider OSS, less frequently outsourcing
 Mature companies with large BUs more often consider
outsourcing
• But several mature companies also consider OSS

18. Supported byDeveloped by
RQ2:
Decision Process?

19. Supported byDeveloped by
Roles involved?

20. Supported byDeveloped by
Systematic process?

21. Supported byDeveloped by
Expert judgment?

22. Supported byDeveloped by
Data-driven?

23. Supported byDeveloped by
Democratic/authoritarian?

24. Supported byDeveloped by
Transparency?

25. Supported byDeveloped by
Analysis/Decision Agreement?

26. Supported byDeveloped by
RQ2
What is the nature of the decision process when
selecting components?
 Ad hoc and systematic decisions equally common
 Expert judgment dominates
• But often based on collected data
 Perceived as democratic/authoritarian equally often
• Agile development with involved developers => democratic
 Developers: “We are involved!” Others: “Not really…”
 Most believe the final decisions agree with analysis
• Developers agree more

27. Supported byDeveloped by
RQ3:
Important qualities?

28. Supported byDeveloped by
Most important -ilities?ISO25010

29. Supported byDeveloped by
Estimation of -ilities?

30. Supported byDeveloped by
Most important input?

31. Supported byDeveloped by
Decision Lead-Time?
Min
Max
Avg

32. Supported byDeveloped by
RQ3
What component qualities are the most
important input to the decision process?
 Functionality suitability is the deal breaker.
• Across all organizations, no matter of CSOs considered.
 Then: reliability, maintainability, performance, security
 Evaluations mainly based on expert judgment
 Estimating performance and reliability takes more time
 Large organizations trust their suppliers more
 Young companies prototype more
• If agile => longer decision time

33. Supported byDeveloped by
Conclusions

34. Supported byDeveloped by
Answers to RQs
RQ1. Which CSOs are typically considered in
industry?
• Typically >1 of: In-house, Outsouce, COTS, and OSS
RQ2. What is the nature of the decision process
when selecting components?
• Differs considerably, but expert judgment dominates
RQ3. What component qualities are the most
important input to the decision process?
• Functional suitability, then reliability (and three more)

35. Supported byDeveloped by
Make-or-buy decisions in software eng. are complex:
• Develop? Buy COTS? Outsource? OSS?
Decisions are mainly based on expert judgment
Functionality deal breaker, then reliability
Flickr:vladislav@munich

36. Supported byDeveloped by
References
 D. Badampudi, C. Wohlin, and K. Petersen. Software component
decision-making: In-house, OSS, COTS or outsourcing - A
systematic literature review. Journal of Systems and Software,
121, pp. 105-124, 2016.
 K. Petersen, D. Badampudi, S. Shah, K. Wnuk, T. Gorschek, E.
Papatheocharous, J. Axelsson, S. Sentilles, I. Crnkovic, and A.
Cicchetti. Choosing Component Origins for Software Intensive
Systems: In-house, COTS, OSS or Outsourcing? - A Case Survey,
IEEE TSE, 2017.
 C. Wohlin, K. Wnuk, D. Smite, U. Franke, D. Badampudi, and A.
Cicchetti. Supporting strategic decision-making for selection of
software assets, ICSOB, 2016.