Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Daniel Méndez
Technical University of Munich, Germany
www.mendezfe.org


Requirements Engineering Research
How good are we...
1968
19771968
19931977
19971993
19971993
19971993
19971993
How relevant is our research to industry?
Example: “Goal-oriented RE” (GORE)
Example: “Goal-oriented RE” (GORE)
Papers published [1]: 966
[1] Horkoff et al. Goal-Oriented Requirements Engineering: A ...
Example: “Goal-oriented RE” (GORE)
Papers published [1]: 966
[1] Horkoff et al. Goal-Oriented Requirements Engineering: A ...
Example: “Goal-oriented RE” (GORE)
Papers published [1]: 966
[2] Mavin, et al. Does Goal-Oriented Requirements Engineering...
Example: “Goal-oriented RE” (GORE)
Papers published [1]: 966
[2] Mavin, et al. Does Goal-Oriented Requirements Engineering...
Example: “Goal-oriented RE” (GORE)
Papers published [1]: 966
[2] Mavin, et al. Does Goal-Oriented Requirements Engineering...
Key question
How can we foster the

practical relevance of our research?
Key question
How can we foster the

practical relevance of our research?
This is a recognised problem
How can we foster the

practical relevance of our research?
What do practitioners think about
How can we foster the

practical relevance of our research?
Of course, we studied also that in more detail.
Franch Vogelsa...
Study design (simplified)
Collection of 435 RE publications 

from 2010-2016
Study design (simplified)
Collection of 435 RE publications 

from 2010-2016
1-sentence summary for each
paper following sa...
Study design (simplified)
Collection of 435 RE publications 

from 2010-2016
1-sentence summary for each
paper following sa...
Study design (simplified)
Collection of 435 RE publications 

from 2010-2016
1-sentence summary for each
paper following sa...
Study design (simplified)
Collection of 435 RE publications 

from 2010-2016
1-sentence summary for each
paper following sa...
Why anonymisation is so important
“Fluffy topic”
“Who cares?”
“Not relevant
for my work”
Eckhardt, Vogelsang, Mendez. 

Ar...
How did practitioners rate the 

relevance of the various research topics?
Rating by contribution topics
Essential
Worthwhile
Unwise
Unimportant
[Blended out for reasons of simplicity]
...
...
Rating by contribution topics
Essential
Worthwhile
Unwise
Unimportant
[Blended out for reasons of simplicity]
Everything s...
Rating by utility / scope facet
Essential
Worthwhile
Unwise
Unimportant
Rating by utility / scope facet
Essential
Worthwhile
Unwise
Unimportant
Even formal methods
are well perceived.
Rating by quality facet
Essential
Worthwhile
Unwise
Unimportant
usability
safety
reliability
performance
security
quality ...
Rating by quality facet
Essential
Worthwhile
Unwise
Unimportant
usability
safety
reliability
performance
security
quality ...
If the topics are not the decisive factor,
what else could matter?
Does the conference venue matter?
Essential
Worthwhile
Unwise
Unimportant
Does the conference venue matter?
Essential
Worthwhile
Unwise
Unimportant
No.
Does the authors’ affiliation matter?
Essential
Worthwhile
Unwise
Unimportant
Does the authors’ affiliation matter?
Essential
Worthwhile
Unwise
Unimportant
No, but…
…the choice of track
seems to be ind...
Does the research method matter?
Essential
Worthwhile
Unwise
Unimportant
Engineering Solution
Proposals
Observational Stud...
Does the research method matter?
Essential
Worthwhile
Unwise
Unimportant
Engineering Solution
Proposals
Observational Stud...
Does the research method matter?
Essential
Worthwhile
Unwise
Unimportant
Engineering Solution
Proposals
Observational Stud...
Key Take-Away
Management Aspects in Requirements Engineering
Agile Requirements Engineering
Human Factors in Requirements ...
Key Take-Away
Management Aspects in Requirements Engineering
Agile Requirements Engineering
Human Factors in Requirements ...
Key Take-Away
Management Aspects in Requirements Engineering
Agile Requirements Engineering
Human Factors in Requirements ...
Upcoming SlideShare
Loading in …5
×

of

Requirements Engineering Research:  How good are we at solving practical problems? Slide 1 Requirements Engineering Research:  How good are we at solving practical problems? Slide 2 Requirements Engineering Research:  How good are we at solving practical problems? Slide 3 Requirements Engineering Research:  How good are we at solving practical problems? Slide 4 Requirements Engineering Research:  How good are we at solving practical problems? Slide 5 Requirements Engineering Research:  How good are we at solving practical problems? Slide 6 Requirements Engineering Research:  How good are we at solving practical problems? Slide 7 Requirements Engineering Research:  How good are we at solving practical problems? Slide 8 Requirements Engineering Research:  How good are we at solving practical problems? Slide 9 Requirements Engineering Research:  How good are we at solving practical problems? Slide 10 Requirements Engineering Research:  How good are we at solving practical problems? Slide 11 Requirements Engineering Research:  How good are we at solving practical problems? Slide 12 Requirements Engineering Research:  How good are we at solving practical problems? Slide 13 Requirements Engineering Research:  How good are we at solving practical problems? Slide 14 Requirements Engineering Research:  How good are we at solving practical problems? Slide 15 Requirements Engineering Research:  How good are we at solving practical problems? Slide 16 Requirements Engineering Research:  How good are we at solving practical problems? Slide 17 Requirements Engineering Research:  How good are we at solving practical problems? Slide 18 Requirements Engineering Research:  How good are we at solving practical problems? Slide 19 Requirements Engineering Research:  How good are we at solving practical problems? Slide 20 Requirements Engineering Research:  How good are we at solving practical problems? Slide 21 Requirements Engineering Research:  How good are we at solving practical problems? Slide 22 Requirements Engineering Research:  How good are we at solving practical problems? Slide 23 Requirements Engineering Research:  How good are we at solving practical problems? Slide 24 Requirements Engineering Research:  How good are we at solving practical problems? Slide 25 Requirements Engineering Research:  How good are we at solving practical problems? Slide 26 Requirements Engineering Research:  How good are we at solving practical problems? Slide 27 Requirements Engineering Research:  How good are we at solving practical problems? Slide 28 Requirements Engineering Research:  How good are we at solving practical problems? Slide 29 Requirements Engineering Research:  How good are we at solving practical problems? Slide 30 Requirements Engineering Research:  How good are we at solving practical problems? Slide 31 Requirements Engineering Research:  How good are we at solving practical problems? Slide 32 Requirements Engineering Research:  How good are we at solving practical problems? Slide 33 Requirements Engineering Research:  How good are we at solving practical problems? Slide 34 Requirements Engineering Research:  How good are we at solving practical problems? Slide 35 Requirements Engineering Research:  How good are we at solving practical problems? Slide 36 Requirements Engineering Research:  How good are we at solving practical problems? Slide 37 Requirements Engineering Research:  How good are we at solving practical problems? Slide 38 Requirements Engineering Research:  How good are we at solving practical problems? Slide 39 Requirements Engineering Research:  How good are we at solving practical problems? Slide 40 Requirements Engineering Research:  How good are we at solving practical problems? Slide 41 Requirements Engineering Research:  How good are we at solving practical problems? Slide 42 Requirements Engineering Research:  How good are we at solving practical problems? Slide 43
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

0 Likes

Share

Download to read offline

Requirements Engineering Research: How good are we at solving practical problems?

Download to read offline

Farewell lecture given for the retirement of Professor Roel Wieringa, University of Twente, March 22nd 2019

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

Requirements Engineering Research: How good are we at solving practical problems?

  1. 1. Daniel Méndez Technical University of Munich, Germany www.mendezfe.org 
 Requirements Engineering Research How good are we at solving practical problems? @mendezfe Farewell Lecture Roel Wieringa Twente University, the Netherlands
  2. 2. 1968
  3. 3. 19771968
  4. 4. 19931977
  5. 5. 19971993
  6. 6. 19971993
  7. 7. 19971993
  8. 8. 19971993
  9. 9. How relevant is our research to industry?
  10. 10. Example: “Goal-oriented RE” (GORE)
  11. 11. Example: “Goal-oriented RE” (GORE) Papers published [1]: 966 [1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016
  12. 12. Example: “Goal-oriented RE” (GORE) Papers published [1]: 966 [1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016 Papers including a case study [1]: 131
  13. 13. Example: “Goal-oriented RE” (GORE) Papers published [1]: 966 [2] Mavin, et al. Does Goal-Oriented Requirements Engineering Achieve its Goal?, 2017 [1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016 Papers including a case study [1]: 131 Studies involving practitioners [2]: 20
  14. 14. Example: “Goal-oriented RE” (GORE) Papers published [1]: 966 [2] Mavin, et al. Does Goal-Oriented Requirements Engineering Achieve its Goal?, 2017 [3] Mendez et al. Naming the Pain in Requirements Engineering Initiative - www.re-survey.org [1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016 Papers including a case study [1]: 131 Studies involving practitioners [2]: 20 Practitioners actually using GORE [3]: ~ 5%
  15. 15. Example: “Goal-oriented RE” (GORE) Papers published [1]: 966 [2] Mavin, et al. Does Goal-Oriented Requirements Engineering Achieve its Goal?, 2017 [3] Mendez et al. Naming the Pain in Requirements Engineering Initiative - www.re-survey.org [1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016 Papers including a case study [1]: 131 Studies involving practitioners [2]: 20 Practitioners actually using GORE [3]: ~ 5% For comparison: Icelanders believing in elves [4]: [4] https://www.nationalgeographic.com/travel/destinations/europe/iceland/believes-elves-exist-mythology/ 54%
  16. 16. Key question How can we foster the
 practical relevance of our research?
  17. 17. Key question How can we foster the
 practical relevance of our research? This is a recognised problem
  18. 18. How can we foster the
 practical relevance of our research? What do practitioners think about
  19. 19. How can we foster the
 practical relevance of our research? Of course, we studied also that in more detail. Franch Vogelsang Knauss Heldal Travassos CarverMéndez Coupette What do practitioners think about
  20. 20. Study design (simplified) Collection of 435 RE publications 
 from 2010-2016
  21. 21. Study design (simplified) Collection of 435 RE publications 
 from 2010-2016 1-sentence summary for each paper following same pattern “A method to automatically recover traceability links between various software artefacts based on topic modelling” “An experiment with students for comparing two requirements elicitation techniques when instantiating software product lines (SPL) in order to understand which approach is more suitable for eliciting requirements in SPL” A [contribution type] to [utility / scope] in order to [improvement goal]
  22. 22. Study design (simplified) Collection of 435 RE publications 
 from 2010-2016 1-sentence summary for each paper following same pattern Online survey with practitioners: – assess the papers based on (sample) summaries – propose research topics they find relevant – demographics
  23. 23. Study design (simplified) Collection of 435 RE publications 
 from 2010-2016 1-sentence summary for each paper following same pattern Online survey with practitioners: – assess the papers based on (sample) summaries – propose research topics they find relevant – demographics • 2164 ratings (positive and negative) and reasons • 103 suggestions on RE research topics
  24. 24. Study design (simplified) Collection of 435 RE publications 
 from 2010-2016 1-sentence summary for each paper following same pattern Online survey with practitioners: – assess the papers based on (sample) summaries – propose research topics they find relevant – demographics • 2164 ratings (positive and negative) and reasons • 103 suggestions on RE research topics Aggregation, anonymisation, analysis, disclosure (open data)
  25. 25. Why anonymisation is so important “Fluffy topic” “Who cares?” “Not relevant for my work” Eckhardt, Vogelsang, Mendez. 
 Are “non-functional” requirements really non- functional? An Investigation of Non-functional Requirements in Practice. (ICSE 2016) Racheva, Daneva, Sikkel Herrmann, Wieringa. 
 Do We Know Enough about Requirements Prioritization in Agile Projects: Insights from a Case Study. (RE 2010) “blablabla” Rated unimportant Rated worthwhile
  26. 26. How did practitioners rate the 
 relevance of the various research topics?
  27. 27. Rating by contribution topics Essential Worthwhile Unwise Unimportant [Blended out for reasons of simplicity] ... ...
  28. 28. Rating by contribution topics Essential Worthwhile Unwise Unimportant [Blended out for reasons of simplicity] Everything seems to be relatively interesting. (To some, even goal models seem essential.) ... ...
  29. 29. Rating by utility / scope facet Essential Worthwhile Unwise Unimportant
  30. 30. Rating by utility / scope facet Essential Worthwhile Unwise Unimportant Even formal methods are well perceived.
  31. 31. Rating by quality facet Essential Worthwhile Unwise Unimportant usability safety reliability performance security quality in general functionality sustainability [Blended out for reasons of simplicity] [Blended out for reasons of simplicity]
  32. 32. Rating by quality facet Essential Worthwhile Unwise Unimportant usability safety reliability performance security quality in general functionality sustainability [Blended out for reasons of simplicity] [Blended out for reasons of simplicity] Every quality aspect seems to matter. Safety, Security, and Reliability seem particularly important.
  33. 33. If the topics are not the decisive factor, what else could matter?
  34. 34. Does the conference venue matter? Essential Worthwhile Unwise Unimportant
  35. 35. Does the conference venue matter? Essential Worthwhile Unwise Unimportant No.
  36. 36. Does the authors’ affiliation matter? Essential Worthwhile Unwise Unimportant
  37. 37. Does the authors’ affiliation matter? Essential Worthwhile Unwise Unimportant No, but… …the choice of track seems to be indicative.
  38. 38. Does the research method matter? Essential Worthwhile Unwise Unimportant Engineering Solution Proposals Observational Studies Controlled Laboratory
 Experiments [Rest blended out 
 for reasons of 
 simplicity] … …
  39. 39. Does the research method matter? Essential Worthwhile Unwise Unimportant Engineering Solution Proposals Observational Studies Controlled Laboratory
 Experiments Yes! Real-world environments strongly preferred over artificial (lab) environments. [Rest blended out 
 for reasons of 
 simplicity] … …
  40. 40. Does the research method matter? Essential Worthwhile Unwise Unimportant Engineering Solution Proposals Observational Studies Controlled Laboratory
 Experiments Yes! Real-world environments strongly preferred over artificial (lab) environments. [Rest blended out 
 for reasons of 
 simplicity] … …
  41. 41. Key Take-Away Management Aspects in Requirements Engineering Agile Requirements Engineering Human Factors in Requirements Engineering ….…. ….…. ‣ Every topic finds its champion in practice ‣ What matters most is how research is carried out: 
 We need to go out of the lab into the wild!
  42. 42. Key Take-Away Management Aspects in Requirements Engineering Agile Requirements Engineering Human Factors in Requirements Engineering ….…. ….…. ‣ Every topic finds its champion in practice ‣ What matters most is how research is carried out: 
 We need to go out of the lab into the wild! Told you so: “The only way to leave idealised assumptions of the laboratory is to enter the real world” Wieringa and Morali. Technical Action Research as a Validation Method in Information Systems Design Science, 2012.
  43. 43. Key Take-Away Management Aspects in Requirements Engineering Agile Requirements Engineering Human Factors in Requirements Engineering ….…. ….…. ‣ Every topic finds its champion in practice ‣ What matters most is how research is carried out: 
 We need to go out of the lab into the wild! Thank you!

Farewell lecture given for the retirement of Professor Roel Wieringa, University of Twente, March 22nd 2019

Views

Total views

184

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

7

Shares

0

Comments

0

Likes

0

×