Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Requirements Engineering Research: How good are we at solving practical problems?

22 views

Published on

Farewell lecture given for the retirement of Professor Roel Wieringa, University of Twente, March 22nd 2019

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Requirements Engineering Research: How good are we at solving practical problems?

  1. 1. Daniel Méndez Technical University of Munich, Germany www.mendezfe.org 
 Requirements Engineering Research How good are we at solving practical problems? @mendezfe Farewell Lecture Roel Wieringa Twente University, the Netherlands
  2. 2. 1968
  3. 3. 19771968
  4. 4. 19931977
  5. 5. 19971993
  6. 6. 19971993
  7. 7. 19971993
  8. 8. 19971993
  9. 9. How relevant is our research to industry?
  10. 10. Example: “Goal-oriented RE” (GORE)
  11. 11. Example: “Goal-oriented RE” (GORE) Papers published [1]: 966 [1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016
  12. 12. Example: “Goal-oriented RE” (GORE) Papers published [1]: 966 [1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016 Papers including a case study [1]: 131
  13. 13. Example: “Goal-oriented RE” (GORE) Papers published [1]: 966 [2] Mavin, et al. Does Goal-Oriented Requirements Engineering Achieve its Goal?, 2017 [1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016 Papers including a case study [1]: 131 Studies involving practitioners [2]: 20
  14. 14. Example: “Goal-oriented RE” (GORE) Papers published [1]: 966 [2] Mavin, et al. Does Goal-Oriented Requirements Engineering Achieve its Goal?, 2017 [3] Mendez et al. Naming the Pain in Requirements Engineering Initiative - www.re-survey.org [1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016 Papers including a case study [1]: 131 Studies involving practitioners [2]: 20 Practitioners actually using GORE [3]: ~ 5%
  15. 15. Example: “Goal-oriented RE” (GORE) Papers published [1]: 966 [2] Mavin, et al. Does Goal-Oriented Requirements Engineering Achieve its Goal?, 2017 [3] Mendez et al. Naming the Pain in Requirements Engineering Initiative - www.re-survey.org [1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016 Papers including a case study [1]: 131 Studies involving practitioners [2]: 20 Practitioners actually using GORE [3]: ~ 5% For comparison: Icelanders believing in elves [4]: [4] https://www.nationalgeographic.com/travel/destinations/europe/iceland/believes-elves-exist-mythology/ 54%
  16. 16. Key question How can we foster the
 practical relevance of our research?
  17. 17. Key question How can we foster the
 practical relevance of our research? This is a recognised problem
  18. 18. How can we foster the
 practical relevance of our research? What do practitioners think about
  19. 19. How can we foster the
 practical relevance of our research? Of course, we studied also that in more detail. Franch Vogelsang Knauss Heldal Travassos CarverMéndez Coupette What do practitioners think about
  20. 20. Study design (simplified) Collection of 435 RE publications 
 from 2010-2016
  21. 21. Study design (simplified) Collection of 435 RE publications 
 from 2010-2016 1-sentence summary for each paper following same pattern “A method to automatically recover traceability links between various software artefacts based on topic modelling” “An experiment with students for comparing two requirements elicitation techniques when instantiating software product lines (SPL) in order to understand which approach is more suitable for eliciting requirements in SPL” A [contribution type] to [utility / scope] in order to [improvement goal]
  22. 22. Study design (simplified) Collection of 435 RE publications 
 from 2010-2016 1-sentence summary for each paper following same pattern Online survey with practitioners: – assess the papers based on (sample) summaries – propose research topics they find relevant – demographics
  23. 23. Study design (simplified) Collection of 435 RE publications 
 from 2010-2016 1-sentence summary for each paper following same pattern Online survey with practitioners: – assess the papers based on (sample) summaries – propose research topics they find relevant – demographics • 2164 ratings (positive and negative) and reasons • 103 suggestions on RE research topics
  24. 24. Study design (simplified) Collection of 435 RE publications 
 from 2010-2016 1-sentence summary for each paper following same pattern Online survey with practitioners: – assess the papers based on (sample) summaries – propose research topics they find relevant – demographics • 2164 ratings (positive and negative) and reasons • 103 suggestions on RE research topics Aggregation, anonymisation, analysis, disclosure (open data)
  25. 25. Why anonymisation is so important “Fluffy topic” “Who cares?” “Not relevant for my work” Eckhardt, Vogelsang, Mendez. 
 Are “non-functional” requirements really non- functional? An Investigation of Non-functional Requirements in Practice. (ICSE 2016) Racheva, Daneva, Sikkel Herrmann, Wieringa. 
 Do We Know Enough about Requirements Prioritization in Agile Projects: Insights from a Case Study. (RE 2010) “blablabla” Rated unimportant Rated worthwhile
  26. 26. How did practitioners rate the 
 relevance of the various research topics?
  27. 27. Rating by contribution topics Essential Worthwhile Unwise Unimportant [Blended out for reasons of simplicity] ... ...
  28. 28. Rating by contribution topics Essential Worthwhile Unwise Unimportant [Blended out for reasons of simplicity] Everything seems to be relatively interesting. (To some, even goal models seem essential.) ... ...
  29. 29. Rating by utility / scope facet Essential Worthwhile Unwise Unimportant
  30. 30. Rating by utility / scope facet Essential Worthwhile Unwise Unimportant Even formal methods are well perceived.
  31. 31. Rating by quality facet Essential Worthwhile Unwise Unimportant usability safety reliability performance security quality in general functionality sustainability [Blended out for reasons of simplicity] [Blended out for reasons of simplicity]
  32. 32. Rating by quality facet Essential Worthwhile Unwise Unimportant usability safety reliability performance security quality in general functionality sustainability [Blended out for reasons of simplicity] [Blended out for reasons of simplicity] Every quality aspect seems to matter. Safety, Security, and Reliability seem particularly important.
  33. 33. If the topics are not the decisive factor, what else could matter?
  34. 34. Does the conference venue matter? Essential Worthwhile Unwise Unimportant
  35. 35. Does the conference venue matter? Essential Worthwhile Unwise Unimportant No.
  36. 36. Does the authors’ affiliation matter? Essential Worthwhile Unwise Unimportant
  37. 37. Does the authors’ affiliation matter? Essential Worthwhile Unwise Unimportant No, but… …the choice of track seems to be indicative.
  38. 38. Does the research method matter? Essential Worthwhile Unwise Unimportant Engineering Solution Proposals Observational Studies Controlled Laboratory
 Experiments [Rest blended out 
 for reasons of 
 simplicity] … …
  39. 39. Does the research method matter? Essential Worthwhile Unwise Unimportant Engineering Solution Proposals Observational Studies Controlled Laboratory
 Experiments Yes! Real-world environments strongly preferred over artificial (lab) environments. [Rest blended out 
 for reasons of 
 simplicity] … …
  40. 40. Does the research method matter? Essential Worthwhile Unwise Unimportant Engineering Solution Proposals Observational Studies Controlled Laboratory
 Experiments Yes! Real-world environments strongly preferred over artificial (lab) environments. [Rest blended out 
 for reasons of 
 simplicity] … …
  41. 41. Key Take-Away Management Aspects in Requirements Engineering Agile Requirements Engineering Human Factors in Requirements Engineering ….…. ….…. ‣ Every topic finds its champion in practice ‣ What matters most is how research is carried out: 
 We need to go out of the lab into the wild!
  42. 42. Key Take-Away Management Aspects in Requirements Engineering Agile Requirements Engineering Human Factors in Requirements Engineering ….…. ….…. ‣ Every topic finds its champion in practice ‣ What matters most is how research is carried out: 
 We need to go out of the lab into the wild! Told you so: “The only way to leave idealised assumptions of the laboratory is to enter the real world” Wieringa and Morali. Technical Action Research as a Validation Method in Information Systems Design Science, 2012.
  43. 43. Key Take-Away Management Aspects in Requirements Engineering Agile Requirements Engineering Human Factors in Requirements Engineering ….…. ….…. ‣ Every topic finds its champion in practice ‣ What matters most is how research is carried out: 
 We need to go out of the lab into the wild! Thank you!

×