Talk given at the International Symposium on Empirical Software Engineering and Measurement

1. Technische Universität München
Where do we stand in Requirements Engineering
Improvement Today?
First Results from a Mapping Study
Joint work with
S. Ognawala,Technische Universität München
M. Daneva, University of Twente
S.Wagner, University of Stuttgart
Daniel Mendez
Technische Universität München
Germany
ESEM 2014
Torino, Italy
@mendezfe

2. Notion of RE quality and its improvement
Socio-economic context
RE “Best Practice” Norm
Goals,
expectations,
…
1. Solution orientation
(Also:“normative”,“prescriptive”)
2
2. Problem orientation
(Also:“Inductive”)
Paradigms (simplified)
A
A. Activity orientation
B
B. Artefact orientation
Serves as
Orientation
Steer
Assess/Benchmark
RE reference modelAdopt
RE improvement principles

3. Paradigms and principles
The ugly truth remains...
Problem:
• Little knowledge about the
• state of the art in Requirements Engineering improvement approaches
• state of empirical evidence
Objectives: Explore the publication space
Weapon of choice: Systematic mapping study

4. RE improvement today
Research questions
RQ1. Of what type is the research?
4
RQ2. Which process improvement phases are considered?
RQ3. What paradigms do the publications focus on?
RQ4. Are the underlying principles of normative or of problem-driven nature?

5. Study design
Overview
RQ1. Of what type is the research?
5
RQ2. Which process improvement phases are considered?
RQ3. What paradigms do the publications focus on?
Interested in the protocol? Read the paper… ;-)
RQ4. Are the underlying principles of normative or of problem-driven nature?

6. Study design
Voting procedure
RQ1. Of what type is the research?
6
RQ2. Which process improvement phases are considered?
RQ3. What paradigms do the publications focus on?
RQ4. Are the underlying principles of normative or of problem-driven nature?
Intermediate
Classification
Discussion
Agreement
No
Yes
Agreement level1st stage: 53.4 % (31/58)2nd stage: 72.4 % (42/58)3rd stage: 86.2 % (50/58)4th stage: 100 % (58/58)

7. RE improvement
Results
RQ1. Of what type is the research?
7
RQ2. Which process improvement phases are considered?
RQ3. What paradigms do the publications focus on?
RQ4. Are the underlying principles of normative or of problem-driven nature?
Validation
Evaluation
Philosophical
Opinion
Experience
Exploratory
Solution
1996 1998 2000 2002 2004 2006 2008 2010 2012Analysis Construction REPI-LC Validation
Distribution per YearContribution Phase
1
21 1 1 1 1 1 1
1
1
1
2
3
1
3
3
2
43
5
1
1
1
1
1
1
2
1
1 1
431
9
4
1
1
24
17
2
N/A
Normative
Problem-Driven
Artefact
Orientation
Activity
Orientation
1
1
1
5
3
4
2
1
18
11
1
1
6
3
2 21638

8. 1
21 1 1 1 1 1 1
1
1
1
2
3
1
3
3
2
43
5
1
1
1
1
1
1
2
1
1 1
431
9
4
1
1
24
17
2
RE improvement
Conclusions
8
Validation
Evaluation
Philosophical
Opinion
Experience
Exploratory
Solution
1996 1998 2000 2002 2004 2006 2008 2010 2Analysis Construction REPI-LC Validation
Distribution per YearContribution Phase
N/A
Normative
Problem-Driven
Artefact
Orientation
Activity
Orientation
1
1
1
5
3
4
2
1
18
11
1
1
6
3
2 21638
• “Healthy” distribution of approaches with many concepts
• Most papers focus on activity-based and normative approaches
» Focus on assessment against activity-based best practice norms
» Triggered by “best practice movement”?
» Little known about benefits and limitations of
» available improvement principles
» available paradigms

9. 1
21 1 1 1 1 1 1
1
1
1
2
3
1
3
3
2
43
5
1
1
1
1
1
1
2
1
1 1
431
9
4
1
1
24
17
2
What now?
9
Validation
Evaluation
Philosophical
Opinion
Experience
Exploratory
Solution
1996 1998 2000 2002 2004 2006 2008 2010 2Analysis Construction REPI-LC Validation
Distribution per YearContribution Phase
N/A
Normative
Problem-Driven
Artefact
Orientation
Activity
Orientation
1
1
1
5
3
4
2
1
18
11
1
1
6
3
2 21638
• First indicators for reluctance against activity-based normative REPI*
» How (and why) to improve RE in a
» holistic manner
» problem-driven
» artefact-based
… manner?
„I am not convinced of the benefits of external standards.“
* http://re-survey.org

10. 10
• Explore principles of artefact-based and problem-driven REPI
• Determine reliable measurements of improvement success
• Evaluate improvement principles in comparative manner
Future research

11. You are cordially invited to join us!
Daniel Méndez
Daniel.Mendez@tum.de
@mendezfe
Thank you!