Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Theory Building in RE - The NaPiRE Initiative

210 views

Published on

Talk I gave on the "Naming the Pain in Requirements Engineering" initiative (www.re-survey.org) at the Seminar on Forty Years of Requirements Engineering – Looking Forward and Looking Back (RE@40) in Kappel am Albis, Switzerland

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Theory Building in RE - The NaPiRE Initiative

  1. 1. Technische Universität München Theory building in RE - 
 The NaPiRE* Initiative * Naming the Pain in Requirements Engineering 
 (www.re-survey.org) Daniel Méndez Technical University of Munich www.mendezfe.org @mendezfe RE@40 Seminar, 2017 Kappel am Albis, Switzerland
  2. 2. “[…] judging a theory by assessing the number, faith, and vocal energy of its supporters […] basic political credo of contemporary religious maniacs” — Imre Lakatos, 1970
  3. 3. ‣ In Software Engineering, too, truth often lies in power rather than in empirical evidence. “[…] judging a theory by assessing the number, faith, and vocal energy of its supporters […] basic political credo of contemporary religious maniacs” — Imre Lakatos, 1970
  4. 4. RE research and practice often dominated by conventional wisdom For a problem-driven research, we nee a 
 reliable body of knowledge about 
 industrial practices and problems.
  5. 5. Naming the Pain in 
 Requirements Engineering (NaPiRE) ‣Foundation for problem-driven research Overall goal: (First) Theory on industrial practices and problems in Requirements Engineering
  6. 6. 2017 22 countries 51 researchers from 43 institutions 2014 10 countries 23 researchers 2012 1 country, 2 researchers Overall approach: By the community for the community Bi-yearly replicated, globally distributed family of surveys (w/ synthesis) on practices and problems ‣ Collaborative instrument design ‣ Independent surveys in different countries ‣ Collaborative synthesis and publication www.re-survey.org Principles ‣ Theory-centric ‣ Anonymity, but closed ‣ Full open science practices
  7. 7. 2017 22 countries 51 researchers from 43 institutions 2014 10 countries 23 researchers Overall approach: By the community for the community Bi-yearly replicated, globally distributed family of surveys (w/ synthesis) on practices and problems ‣ Collaborative instrument design ‣ Independent surveys in different countries ‣ Collaborative synthesis and publication www.re-survey.org Principles ‣ Theory-centric ‣ Anonymity, but closed ‣ Full open science practices
  8. 8. 2017 22 countries 51 researchers from 43 institutions Overall approach: By the community for the community Bi-yearly replicated, globally distributed family of surveys (w/ synthesis) on practices and problems ‣ Collaborative instrument design ‣ Independent surveys in different countries ‣ Collaborative synthesis and publication www.re-survey.org Principles ‣ Theory-centric ‣ Anonymity, but closed ‣ Full open science practices
  9. 9. Excerpt results NapiRE 2015/16 Country # Companies
 (full survey completion) Response rate Austria 14 72 % Germany 41 36,8 % UK (Northern Ireland) 18 39,7 % Canada 13 75 % USA 15 36,2 % Estonia 8 25,7 % Finland 15 37,5 % Norway 10 70,8 % Sweden 20 51,8 % Brazil 74 35,3 % Total 228 crosscutting (60) public (28) sector (28) enterprise (20) finance(26) planning (20) resource (23) automotive (15) healthcare(14) management (16) business (9) energy (11) insurance(11) logistics (8) telecom(9) transportation(8) aerospace (5) education (5) gas (5) geoprocessing (4) informed(5) intelligence (4) oil (5) process (5) aviation (3) communication (3) construction (3) defence (3) e-commerce (3) entertainment (3) games (3) human (3) shipping (3) workforce (3) agriculture (2) chemicals (2) customer (2) electronic (2) railway (2) relationship (2) scientific (2) software (2) automation (1) e- government (1) funerary (1) goods (1) industrial (1) networking (1) pulp (1) steel (1) Distributed over industry sectors
  10. 10. Status quo in practices used [1] ‣ Requirements Elicitation ‣ Requirements Documentation ‣ Requirements Change and Alignment ‣ Requirements Engineering Standards ‣ Requirements Engineering Improvement Problems in RE [2] ‣ Problems and criticality ‣ Causes ‣ Effects [2] Mendez, Wagner, et al. Naming the pain in requirements engineering Contemporary problems, causes, and effects in practice , 2017 [1] Wagner, Mendez, et al. Status Quo in Requirements Engineering: A Theory and a Global Family of Surveys, 2017 Req Elicitation Technique Interview Scenario Prototyping Facilitated Meetings Observation Req Documentation Technique Structured req list Domain/business process model Use case model Goal model Data model Non-functional req Textual Semi-formal Formal Technology Req Test Alignment Approach Req review by tester Coverage by tests Acceptance criteria Test derivation from models Req Change Approach Product backlog update Change requests Trace management Impact analysis Activity Req Elicitation Req Documentation Req Change Management Req Test Alignment P 1-5 P 6-13 P 14-20 P 21-24 Actor Req Engineer Test Engineer Req Standard Application Practice Control Tailoring Req Eng Process Standard P 25-28 Req Standard Defintion Compliance Development Tool support Quality assurance Project management Knowledge transfer Process complexity Communication demand Willigness to change Possibility of standardisation P 26-45 Req Improvement Means Continuous improvement Strengths/weaknesses Own business unit/role Req Eng ImprovementP 46--48 Excerpt results NapiRE 2015/16
  11. 11. Status quo in practices used [1] ‣ Requirements Elicitation ‣ Requirements Documentation ‣ Requirements Change and Alignment ‣ Requirements Engineering Standards ‣ Requirements Engineering Improvement Problems in RE [2] ‣ Problems and criticality ‣ Causes ‣ Effects [2] Mendez, Wagner, et al. Naming the pain in requirements engineering Contemporary problems, causes, and effects in practice , 2017 [1] Wagner, Mendez, et al. Status Quo in Requirements Engineering: A Theory and a Global Family of Surveys, 2017 Req Elicitation Technique Interview Scenario Prototyping Facilitated Meetings Observation Req Documentation Technique Structured req list Domain/business process model Use case model Goal model Data model Non-functional req Textual Semi-formal Formal Technology Req Test Alignment Approach Req review by tester Coverage by tests Acceptance criteria Test derivation from models Req Change Approach Product backlog update Change requests Trace management Impact analysis Activity Req Elicitation Req Documentation Req Change Management Req Test Alignment P 1-5 P 6-13 P 14-20 P 21-24 Actor Req Engineer Test Engineer Req Standard Application Practice Control Tailoring Req Eng Process Standard P 25-28 Req Standard Defintion Compliance Development Tool support Quality assurance Project management Knowledge transfer Process complexity Communication demand Willigness to change Possibility of standardisation P 26-45 Req Improvement Means Continuous improvement Strengths/weaknesses Own business unit/role Req Eng ImprovementP 46--48 Excerpt results NapiRE 2015/16 Teaser
  12. 12. In scope of artefact-based standard (reference) models
  13. 13. In scope of artefact-based standard (reference) models
  14. 14. Unclear roles and responsabilities at customer side Weak qualification of RE team members Subjective interpretations Stakeholders lack business vision and understanding (1.28%) Customer does not know what he wants (6.41%) Lack of experience of RE team members Complexity of domain Insufficient resources Lack of time Conflicting stakeholder viewpoints (8.97%) Strict time schedule by customer Missing req. specification template Missing IT project experience at customer side Weak qualification of stakeholders (6.41%) Input (33%) (3.85%) (6.41%) (2.56%) (1.28% ) (1.28%) (1.28%) (2.56%) (10.26 %) (2.56%) (2.56%) (1.28%) (1.28%)(1.28%) (1.10%) Method (38%) Req. too abstract Missing completeness check of requirements Insufficient agility Communication flaws between team and customer Lack of a well-defined RE process Complexity of RE Unclear terminology (11.54%) (1.28%)(1.28%) (2.56%) (2.56%) (1.28%) (1.28%) People (17%) Tools (3%) (1.28%) Communication flaws customer Missing engagement by customer Missing customer involvement (2.56%) (1.28%) (1.28%) Policy restrictions (1.28%) (1.28%) Missing RE awareness at customer side (1.28%) Weak management at customer side (1.28%) Missing direct communication to customer Missing involvement of dev. (2.56%) Poor project management Missing solution approach Organization (9%) Too high team distribution Many customer s Language Barriers Not following the communication plan Missing tool support(1.28%) Customer (19%) Design or Implementation (9%) Product (22%) Validation or Verification (3%) Increased number of req. changes (16.18%) Incomplete Req. (2.94% ) Decreased user acceptance Customer dissatisfaction (2.94%) Need for post implementation Poor product quality Decreased business value (10.29%) (8.82%) (2.94%) Increased communication Implementation of irrelevant req. Effort overrun Time overrun Budget overrun Decreased efficiency (overall) Increased number of change requests Wrong estimates Increased difficulty of req. elicitation Validation of req. becomes difficult Project or Organization (47%) (2.94%) (1.47%) (13.24%) (5.88%) (4.41%) (2.94%)(2.94%) (2.94%) (1.47%) (1.47%) (1.47%) (1.47%) (2.94%) Poor (system) design quality (1.47%) Misunderstanding (overall) Late decision (2.94%) (2.94%) (2.94%) Overall demotivation Information gets lost Increased complexity in RE
  15. 15. ) Communication flaws customer Customer (19%) Design or Implementation (9%) Product (22%) Validation or Verification (3%) Increased number of req. changes (16.18%) Incomplete Req. (2.94% ) Decreased user acceptance Customer dissatisfaction (2.94%) Need for post implementation Poor product quality Decreased business value (10.29%) (8.82%) (2.94%) Increased communication Implementation of irrelevant req. Effort overrun Time overrun Budget overrun Decreased efficiency (overall) Increased number of change requests Wrong estimates Increased difficulty of req. elicitation Validation of req. becomes difficult Project or Organization (47%) (2.94%) (1.47%) (13.24%) (5.88%) (4.41%) (2.94%)(2.94%) (2.94%) (1.47%) (1.47%) (1.47%) (1.47%) (2.94%) Poor (system) design quality (1.47%) Misunderstanding (overall) Late decision (2.94%) (2.94%) (2.94%) Overall demotivation Information gets lost Increased complexity in RE
  16. 16. Unclear roles and responsabilities at customer side Weak qualification of RE team members Subjective interpretations Stakeholders lack business vision and understanding (1.28%) Customer does not know what he wants (6.41%) Lack of experience of RE team members Complexity of domain Insufficient resources Lack of time Conflicting stakeholder viewpoints (8.97%) Strict time schedule by customer Missing req. specification template Missing IT project experience at customer side Weak qualification of stakeholders (6.41%) Input (33%) (3.85%) (6.41%) (2.56%) (1.28% ) (1.28%) (1.28%) (2.56%) (10.26 %) (2.56%) (2.56%) (1.28%) (1.28%)(1.28%) (1.10%) Method (38%) Req. too abstract Missing completeness check of requirements Insufficient agility Communication flaws between team and customer Lack of a well-defined RE process Complexity of RE Unclear terminology (11.54%) (1.28%)(1.28%) (2.56%) (2.56%) (1.28%) (1.28%) People (17%) Tools (3%) (1.28%) Communication flaws customer Missing engagement by customer Missing customer involvement (2.56%) (1.28%) (1.28%) Policy restrictions (1.28%) (1.28%) Missing RE awareness at customer side (1.28%) Weak management at customer side (1.28%) Missing direct communication to customer Missing involvement of dev. (2.56%) Poor project management Missing solution approach Organization (9%) Too high team distribution Many customer s Language Barriers Not following the communication plan Missing tool support(1.28%)
  17. 17. Unclear roles and responsabilities at customer side Weak qualification of RE team members Subjective interpretations Stakeholders lack business vision and understanding (1.28%) Customer does not know what he wants (6.41%) Lack of experience of RE team members Complexity of domain Insufficient resources Lack of time Conflicting stakeholder viewpoints (8.97%) Strict time schedule by customer Missing req. specification template Missing IT project experience at customer side Weak qualification of stakeholders (6.41%) Input (33%) (3.85%) (6.41%) (2.56%) (1.28% ) (1.28%) (1.28%) (2.56%) (10.26 %) (2.56%) (2.56%) (1.28%) (1.28%)(1.28%) (1.10%) Method (38%) Req. too abstract Missing completeness check of requirements Insufficient agility Communication flaws between team and customer Lack of a well-defined RE process Complexity of RE Unclear terminology (11.54%) (1.28%)(1.28%) (2.56%) (2.56%) (1.28%) (1.28%) People (17%) Tools (3%) (1.28%) Communication flaws customer Missing engagement by customer Missing customer involvement (2.56%) (1.28%) (1.28%) Policy restrictions (1.28%) (1.28%) Missing RE awareness at customer side (1.28%) Weak management at customer side (1.28%) Missing direct communication to customer Missing involvement of dev. (2.56%) Poor project management Missing solution approach Organization (9%) Too high team distribution Many customer s Language Barriers Not following the communication plan Missing tool support(1.28%) Causes rather not due to missing guidance via reference models, but due to organisational and even social issues.
  18. 18. Do teams applying agile practices avoid these issues? Unclear roles and responsabilities at customer side Weak qualification of RE team members Subjective interpretations Stakeholders lack business vision and understanding (1.28%) Customer does not know what he wants (6.41%) Lack of experience of RE team members Complexity of domain Insufficient resources Lack of time Conflicting stakeholder viewpoints (8.97%) Strict time schedule by customer Missing req. specification template Missing IT project experience at customer side Weak qualification of stakeholders (6.41%) Input (33%) (3.85%) (6.41%) (2.56%) (1.28% ) (1.28%) (1.28%) (2.56%) (10.26 %) (2.56%) (2.56%) (1.28%) (1.28%)(1.28%) (1.10%) Method (38%) Req. too abstract Missing completeness check of requirements Insufficient agility Communication flaws between team and customer Lack of a well-defined RE process Complexity of RE Unclear terminology (11.54%) (1.28%)(1.28%) (2.56%) (2.56%) (1.28%) (1.28%) People (17%) Tools (3%) (1.28%) Communication flaws customer Missing engagement by customer Missing customer involvement (2.56%) (1.28%) (1.28%) Policy restrictions (1.28%) (1.28%) Missing RE awareness at customer side (1.28%) Weak management at customer side (1.28%) Missing direct communication to customer Missing involvement of dev. (2.56%) Poor project management Missing solution approach Organization (9%) Too high team distribution Many customer s Language Barriers Not following the communication plan Missing tool support(1.28%) Customer (19%) Design or Implementation (9%) Product (22%) Validation or Verification (3%) Increased number of req. changes (16.18%) Incomplete Req. (2.94% ) Decreased user acceptance Customer dissatisfaction (2.94%) Need for post implementation Poor product quality Decreased business value (10.29%) (8.82%) (2.94%) Increased communication Implementation of irrelevant req. Effort overrun Time overrun Budget overrun Decreased efficiency (overall) Increased number of change requests Wrong estimates Increased difficulty of req. elicitation Validation of req. becomes difficult Project or Organization (47%) (2.94%) (1.47%) (13.24%) (5.88%) (4.41%) (2.94%)(2.94%) (2.94%) (1.47%) (1.47%) (1.47%) (1.47%) (2.94%) Poor (system) design quality (1.47%) Misunderstanding (overall) Late decision (2.94%) (2.94%) (2.94%) Overall demotivation Information gets lost Increased complexity in RE
  19. 19. Small companies
 (1-50) Medium companies
 (51-250) Large companies
 (> 251) #1 Problem Incomplete/hidden requirements Communication flaws customer
 Incomplete/hidden requirements #2 Problem Communication flaws customer
 Incomplete/hidden requirements Moving targets #3 Problem Underspecified requirements Communication flaws project team Communication flaws customer
 #4 Problem Communication flaws project team
 Separation reqs. / solutions Time boxing #5 Problem Time boxing Weak access customer needs Underspecified requirements Do teams applying agile practices avoid these issues?
  20. 20. Small companies
 (1-50) Medium companies
 (51-250) Large companies
 (> 251) #1 Problem Incomplete/hidden requirements Communication flaws customer
 Incomplete/hidden requirements #2 Problem Communication flaws customer
 Incomplete/hidden requirements Moving targets #3 Problem Underspecified requirements Communication flaws project team Communication flaws customer
 #4 Problem Communication flaws project team
 Separation reqs. / solutions Time boxing #5 Problem Time boxing Weak access customer needs Underspecified requirements No. Do teams applying agile practices avoid these issues?
  21. 21. Small companies
 (1-50) Medium companies
 (51-250) Large companies
 (> 251) #1 Problem Incomplete/hidden requirements Communication flaws customer
 Incomplete/hidden requirements #2 Problem Communication flaws customer
 Incomplete/hidden requirements Moving targets #3 Problem Underspecified requirements Communication flaws project team Communication flaws customer
 #4 Problem Communication flaws project team
 Separation reqs. / solutions Time boxing #5 Problem Time boxing Weak access customer needs Underspecified requirements Not at all. Do teams applying agile practices avoid these issues?
  22. 22. How can we make use of such insights?
  23. 23. How can we make use of such insights? ‣ Support problem-driven research
  24. 24. How can we make use of such insights? Raw data ‣ Anonymised ‣ Sanitised ‣ Curated (w/ codebooks) ‣ Support problem-driven research
  25. 25. How can we make use of such insights? Open data repository tera-PROMISE 
 http://openscience.us/repo/
 (will probably change) Raw data ‣ Anonymised ‣ Sanitised ‣ Curated (w/ codebooks) ‣ Support problem-driven research
  26. 26. How can we make use of such insights? Open data repository tera-PROMISE 
 http://openscience.us/repo/
 (will probably change) Raw data ‣ Anonymised ‣ Sanitised ‣ Curated (w/ codebooks) Problem-driven research supports ‣ Support problem-driven research
  27. 27. Phenomena and measurements
  28. 28. How can we make use of such insights? ‣ Immediate operationalisation for practical environments
  29. 29. How can we make use of such insights? ‣ Immediate operationalisation for practical environments Example: 
 evidence-based Risk Management* * Based on two MSc. theses: Priscilla Mafra at UFF, Brazil; Michaela Tießler at TUM, Germany
  30. 30. How can we make use of such insights? 1 Build holistic model with cause-effect relationships ‣ Immediate operationalisation for practical environments
  31. 31. How can we make use of such insights? 1 Build holistic model with cause-effect relationships ‣ Immediate operationalisation for practical environments
  32. 32. Implement Bayesian network 
 including context factors 2 How can we make use of such insights? 1 Build holistic model with cause-effect relationships ‣ Immediate operationalisation for practical environments
  33. 33. How can we make use of such insights? 3 Operationalise as a 
 risk assessment approach ‣ Immediate operationalisation for practical environments Build holistic model with cause-effect relationships1 2 Implement Bayesian network including context factors
  34. 34. How can we make use of such insights? 3 Operationalise as a 
 risk assessment approach ‣ Immediate operationalisation for practical environments Build holistic model with cause-effect relationships1 2 Implement Bayesian network including context factors
  35. 35. How can we make use of such insights? 3 Operationalise as a 
 risk assessment approach ‣ Immediate operationalisation for practical environments Build holistic model with cause-effect relationships1 2 Implement Bayesian network including context factors
  36. 36. How can we make use of such insights? ‣ Immediate operationalisation for practical environments Build holistic model with cause-effect relationships1 2 Implement Bayesian network including context factors 3 Operationalise as a risk assessment approach 4 Offer situation-spec. guidelines 
 (corrective/preventive measures)
  37. 37. Preparation Where do we stand right now? •Preparation instruments about to be completed Next steps •Data disclosure previous runs •Survey implementation & pilot phase next run ‣ Start independent data collections; 
 (before summer break) ISERN 2017 (October) • Discussion 1st Results • Analysis strategy Survey / Data Collection (Coordinated local replications) Survey ++ Data Analysis and Publication (Synthesis, planning, joint publication) We are here... ISERN 2018 • Data analysis 
 and publication (cont.)
  38. 38. You are cordiallyinvited to join us! www.re-survey.org Thank you! Daniel Méndez Technical University of Munich www.mendezfe.org @mendezfe

×