ESEM Summer school, A systematic mapping study about engineering aspects of software startups presented in ESEM summer school 2018, Oulu, Finland

1. Software Startup Engineering: A Systematic Mapping
Study
Anh Nguyen-Duc, Vebjørn Berg, Jørgen Birkeland,
Ilias Pappas, Letizia Jaccheri
EMSE Summer
school 2018
1

2. About myself
• 2011-2014, Ph.D from Norwegian University of
Science and Technology
– PhD title “Supporting coordination of software
development across organizational boundaries”
• 2015 – currently, entrepreneurs, startup
enthusiast, startup researcher
– Software Startup Labs
• 2017 – currently, Associate Professor at University
of South Eastern Norway
– Human Computer Interaction, Project driven
courses

3. Motivation

4. Case - Muml
• Vision: To be the fastest provider of validated breaking news - by
capitalizing on live, super-local user-generated content + validation
technology
• Funding:
– 100k Eur from Innovation Norway,
– 50k Usd from Google News Initiatives
– 50k Nok self-funding
• Outsourced development team in Asia
• A fully functional product
4

5. Case - Muml
5

6. Case - Muml
• Terminated operation after 2 years 7 months 18 days
• Reasons for failures
– Explainable factors
• Human factors
• Financial factors
• Process factors
• Technology factors
– Unexplainable factors
6
Certain failures can be avoid by learning
either from past experiences or systematic
and external knowledge

7. Startups vs. SMEs
Startup companies are unique:
Little or no operating history
Limited resources
Multiple influences
Dynamic technologies and markets
7

8. Motivations
• Startup researchers have called for a further attention
to engineering approaches
• Systematic mapping study helps to identify the current
status in the area and pave the way for more empirical
studies examining startups
8
N. Paternoster, C. Giardino, M. Unterkalmsteiner, T. Gorschek, P. Abra- hamsson, Software
development in startup companies: A systematic map- ping study, Information and Software Technology
56 (10) (2014)
E. Klotins, M. Unterkalmsteiner, T. Gorschek, Software Engineering Knowledge Areas in Startup
Companies: A Mapping Study, Vol. 210 of Lecture Notes in Business Information Processing, 2015, pp.

9. Research Questions
• RQ1: How has software startup research changed over time in
terms of focused knowledge areas?
• RQ2: What is the relative strength of the empirical evidence
reported?
• RQ3: In what context has software startup research been
conducted?
9

10. Research Methodology
10

11. Research Methodology
11

12. Results
RQ1: How has software startup
research changed over time in
terms of focused knowledge areas?
Focus areas are software process,
management, construction, design, and
requirements, with the shift of focus toward
process and management areas.
RQ2: What is the relative strength
of the empirical evidence reported?
The rigor of primary papers was higher
between 2013-2017 than that of 1994-2013.
RQ3: In what context has software
startup research been conducted?
Thematic concepts representing the software
startup context include innovation, lack of
resources, uncertainty, time-pressure, small
team, highly reactive, and rapidly evolving.
12

13. RQ1- Publication Frequency, 1994-2017 (n=74)
13

14. RQ1- SE Knowledge area coverage
14

15. RQ1- SE Process
• The need for adapting software development process to startup context:
– Contextual factors: project’s scope, magnitude, complexity, and changing
requirements
• The lack of guidance how startups can establish their methods
• More studies to contribute to the adoption of agile practices in startups
15

16. RQ1- SE Professional Practices
• Startup processes need to cover both business and engineering aspects
• Startup developers need to acquire not only technical competence but also
business mindset
• Developers in software startups typically prioritize speed related agile practices
rather than quality related ones
• A possible research area is to investigate how universities can facilitate learning and
to support the specific needs of practitioners that are to work in software startups
16

17. RQ1- SE Management
• Relates software project management to business model experimentation and
customer development
• Startups struggle with how pivoting should be performed at diferent lifecycle
stages?
• Startup-aware outsourcing is a feasible option for early-stage startups
• A need for more research to identify how startups explicitly manage risks
17

18. RQ1- Software Requirements
• Requirements mainly were elicited through the founders’ assumptions and
interpretations of the market
• Minimum Viable Products (MVPs) are effective tools for requirements elicitation
• The lack of studies investigating how software startups perform requirements
engineering processes
18

19. RQ1- Software Design
• Requirements mainly were elicited through the founders’ assumptions and
interpretations of the market
• The lack of studies investigating how software startups perform requirements
engineering processes
19

20. RQ1- Software Quality
• Testing is critical to startups’ success, but often overlooked
• The most common testing techniques:
– unit tests (37%)
– pilot clients (25%)
– functional tests (25%)
– specialist testers (13%)
• Future research on how startups can learn from established companies’ systematic
testing processes
20

21. RQ1- Software Construction
• There does not exist a clear understanding of how entrepreneurs can use the
different tools efficiently to meet their specific needs
• A software tool portal can be helpful to support software construction
21

22. RQ2- Type of research publications
22

23. RQ2- Rigour of papers in 2013-2017 (n=27)
23

24. RQ2- Rigour of papers in 1994-2013 (n=47)
24

25. RQ3- Thematic Concepts, 1994-2017
Thematic Concepts
Frequency 13’-
17’ (#27)
Frequency 94’-
13’ (#47)
Innovation/Innovative 15 19
Uncertainty 14 15
Small team 11 12
Lack of resources 9 21
Little working/operating history 9 3
Time-pressure 7 17
Rapidly evolving 5 16
New company 5 8
Highly reactive 3 19
Highly risky 3 8
Third party dependency 2 12
One product 2 9
Not self-sustained 1 3
Low-experienced team 0 9
Flat organisation 0 5
25

26. Final remarks
• Software startups find it hard to apply theory in practice
• Future work on certain research themes, i.e startup evolution models,
human aspects, and consolidation of contextual factors
• Multivocal Literature Review can be the next review on software startup
26