Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CIbSE-RET 2019 keynote

41 views

Published on

The Road towards Data-Driven Requirements Engineering

Published in: Software
  • Be the first to comment

  • Be the first to like this

CIbSE-RET 2019 keynote

  1. 1. The Road towards Data-Driven Requirements Engineering Xavier Franch Universitat Politècnica de Catalunya (UPC-BarcelonaTech) Keynote at CIbSE’19-RET Track, April 2019 La Habana, Cuba
  2. 2. 2 RE (still) central in SE CIbSE-RET2019©XavierFranch
  3. 3. 3 RE (still) a problem in SE Causes of failure CIbSE-RET2019©XavierFranch
  4. 4. 4 Several reasons for this... How to ensure that the system is delivering the right value to stakeholders? [14] [15] ...business value being one of them CIbSE-RET2019©XavierFranch “80% of products lose money due to wrongly set product objectives that result in building products that customers do not need”
  5. 5. 5 From traditional RE... [1] [2] [3] [4] CIbSE-RET2019©XavierFranch
  6. 6. 6 ...to data-driven RE [4] [5] [6][7] [8] [9] CIbSE-RET2019©XavierFranch
  7. 7. 7 The data-driven RE cycle Feedback analysis Usage analysis Decision-making {Requirements} Repository mining [10] Analytics tool CIbSE-RET2019©XavierFranch
  8. 8. 8 CIbSE-RET2019©XavierFranch Example: The Q-Rapids project
  9. 9. 9 (EXPLICIT) FEEDBACK MANAGEMENT CIbSE-RET2019©XavierFranch
  10. 10. 10 (Explicit) Feedback management • Gathering, analysing and summarizing feedback given by the user CIbSE-RET2019©XavierFranch
  11. 11. 11 Gathering user feedback • Communication style ▪ Push vs. pull • Mode ▪ From linguistic to multi-modal • Channel ▪ App stores, forums, social media, … • Advanced features ▪ E.g., feedback to feedback CIbSE-RET2019©XavierFranch
  12. 12. 12 CIbSE-RET2019©XavierFranch Feedback analysis • Categorization • Sentiment analysis • Topic modeling
  13. 13. 13 Behind the curtains: NLP & ML • NLP: processing textual information • ML: every context is different CIbSE-RET2019©XavierFranch
  14. 14. 14 Categorization • Bug report • Feature request • Sometimes difficult to distinguish ▪ “It’s not a bug, it’s a feature” Also: comments, general ideas, just praise, ... [11] [12] CIbSE-RET2019©XavierFranch
  15. 15. 15 Can be more elaborated CIbSE-RET2019©XavierFranch
  16. 16. 16 Sentiment analysis Process of assigning a quantitative value to a piece of text expressing an affect or mood Use of dictionaries: But of course, not easy...: “Great, I love this feature that gives me this lovely headache” CIbSE-RET2019©XavierFranch Models using advanced ML techniques
  17. 17. 17 Topic modeling ▪ Instability problems (“order effects”) ▪ Fails to capture rich topical correlations LDA CIbSE-RET2019©XavierFranch • Identifying topics that best describes a corpus ▪ Usually, latent (emerge during the process) • Each corpus described by a distribution of topics ▪ Each topic described by a distribution of words • Most popular algorithm: Latent Dirichlet Allocation
  18. 18. 18 Preprocessing • From text to lexical/syntactical units ▪ Tokenization ▪ Lemmatization ▪ Stemming ▪ Phrasing • Part-of-speech tagging “I have a problem when saving the document, please check it” I/PRP have/VBP a/DT problem/NN when/WRB saving/VBG the/DT document/NN ,/, please/VBP check/VB it/PRP Speech-art: Requestive CIbSE-RET2019©XavierFranch
  19. 19. 19 Summarization CIbSE-RET2019©XavierFranch
  20. 20. 20 Summarization CIbSE-RET2019©XavierFranch
  21. 21. 21 Challenges with explicit feedback CIbSE-RET2019©XavierFranch
  22. 22. 22 MONITORING (IMPLICIT FEEDBACK) CIbSE-RET2019©XavierFranch
  23. 23. 23 QoS Monitoring CIbSE-RET2019©XavierFranch
  24. 24. 24 Important in domains like IoT CIbSE-RET2019©XavierFranch [13]
  25. 25. 25 Importance of context
  26. 26. 26 Contextual requirements (context, behavior) Drowsy driver Ring alarm Satisfaction: given cr = (ctx, beh) eval(cr) = ctx  dom(Oper)  evaluable(ctx)  ctx  beh Objective function: minimize {cri: 0 < i ≤ m: NOT EvalCR(cri)}
  27. 27. 27 Not so easy... CIbSE-RET2019©XavierFranch
  28. 28. 28 Usage logs • Information about usage of the system • What can be discovered? ▪ Which functionalities are most used ▪ Which navigational paths prevail (or not) ▪ Which calls results often in error codes ▪ … ▪ Especially if QoS also available
  29. 29. 29 COMBINING IMPLICIT AND EXPLICIT FEEDBACK CIbSE-RET2019©XavierFranch
  30. 30. 30 Overall view CIbSE-RET2019©XavierFranch
  31. 31. 31 Architecture CIbSE-RET2019©XavierFranch
  32. 32. 32 Integration ontology CIbSE-RET2019©XavierFranch
  33. 33. 33 Crowd-based RE CIbSE-RET2019©XavierFranch
  34. 34. 34 REPOSITORY MINING CIbSE-RET2019©XavierFranch
  35. 35. 35 Mining repositories CIbSE-RET2019©XavierFranch
  36. 36. 36 From raw data to quality factors ⚫ Focus on internal quality ⚫ Emphasis on modularity CIbSE-RET2019©XavierFranch
  37. 37. 37 1. Definition of metrics Raw metric Size metric Normalized metric Attribute + Data source Business value Data availability Raw metric Data source Code complexity SonarQube Function’s cyclomatic complexity Number of functions Average cyclomatic complexity GitHub Nb. commits CIbSE-RET2019©XavierFranch
  38. 38. 38 2. Evaluation of attributes ⚫ Determination of metrics’ impact onto attributes ◼ Use of (linear) utility functions o Utility: strength of preference a decision-maker has among alternatives regarding a specific quality aspect Norm. metric Attribute Norm. metric Thresholds coming from benchmarking CIbSE-RET2019©XavierFranch
  39. 39. 39 3. Aggregation of factors ⚫ Synthesis of individual utilities into global one ◼ Weights for contributing factors Attribute Attribute Attribute +w1 w2 w3 ... Weights coming from existing data or expert opinion Code complexity Coupling degree Code modifiability CIbSE-RET2019©XavierFranch
  40. 40. 40 4. Interpretation ⚫ Translation of utility values into a QA value ◼ Interpretation model transforming from ratio to ordinal ◼ Could be done at the level of general quality, a characteristic or a subcharacteristic ◼ E.g., analogy with school grade Attribute Subcharac- teristic Attribute v1 v2 Code modifiability API stability Modifiability CIbSE-RET2019©XavierFranch
  41. 41. 41 DECISION MAKING CIbSE-RET2019©XavierFranch
  42. 42. 42 Keys to decision making • Having at hand all information to make informed decisions ▪ Analytics tools • Allowing all stakeholders to have a part in the process ▪ Prioritization CIbSE-RET2019©XavierFranch
  43. 43. 43 Code analytic tools CIbSE-RET2019©XavierFranch Good, but not strategic enough
  44. 44. 44 Strategic dashboards CIbSE-RET2019©XavierFranch
  45. 45. 45 Prediction capabilities CIbSE-RET2019©XavierFranch
  46. 46. 46 Simulation capabilities CIbSE-RET2019©XavierFranch
  47. 47. 47 Decision making CIbSE-RET2019©XavierFranch • Need to involve relevant stakeholders
  48. 48. 48 Liquid democracy CIbSE-RET2019©XavierFranch • Stakeholders can evaluate requirements based on different interest dimensions ▪ Most stakeholders might not be appropiate to evaluate all requirements accurately ▪ Stakeholders can delegate votes to experts selectively
  49. 49. 49 CONCLUSIONS CIbSE-RET2019©XavierFranch
  50. 50. 50 Data-driven RE CIbSE-RET2019©XavierFranch • Offers a great opportunity for delivering more business value to systems’ stakeholders • Some considerations ▪ Not a hammer for every nail ▪ Data-driven → needs data • Still traditional methods at least to start with • The role of traditional RE in the loop is a matter of debate ▪ New, but maybe not so much • E.g., compare to experimentation (from lean startup) – A/B tests, fake door tests, …
  51. 51. 51 Challenges and lessons learned CIbSE-RET2019©XavierFranch • It is not for free • Need to tailor to every company (even project) • Requires expertise • Better if incremental • Transparency is a must
  52. 52. 52 Summary
  53. 53. 53 Images [1] https://dumielauxepices.net/sites/default/files/panels-clipart-focus-group-discussion-706060-4505275.png [2] https://planningshopintl.com/solution/ethnography/ [3] https://www.lrmhc.org/2017-community-health-needs-survey/ [4] http://es.coolclips.com/m/vector/reli0055/Diez-Mandamientos/ [5] https://www.flaticon.com/free-icon/log-file-format_28822 [6] https://www.flaticon.com/free-icon/manager_115893 [7] https://www.iconfinder.com/icons/3356606/computer_geek_glasses_nerd_icon [8] https://www.pinterest.es/pin/802625964810288333/ [9] https://commons.wikimedia.org/wiki/File:Globe_icon.svg [10] https://www.glocksoft.com/email-marketing-software/web-analytics-tools/ [11] https://www.onlinewebfonts.com/icon/488455 [12] https://www.123rf.com/photo_100296089_stock-vector-pray-or-hands-together-in-prayer-line-art-vector-icon-for-religious-apps-and-websites.html [13] http://blog.netumo.com/monitoring-the-internet-of-things-iot/ [14] https://www.onlinefreecourse.net/lynda/requirements-elicitation-for-business-analysts-interviews-lynda-free-download/ [15] https://www.lynda.com/Project-Management-tutorials/Applying-Agile-MoSCoW-Prioritization/718626-2.html All other images labelled for reuse CIbSE-RET2019©XavierFranch
  54. 54. 54 CIbSE-RET2019©XavierFranch
  55. 55. 55 Example: back to Twitter… • Objective: getting the classification tool-supported Tweets contain relevant information for different stakeholders. However, the proportion of relevant information is small compared to the vast amount of received tweets
  56. 56. 56 Example: back to Twitter… • Approach: multi-label classification with a binary relevance method ▪ Assessed classifiers: C4.5 and SVM • Steps ▪ Manually build ground truth (1.000 tweets) ▪ Preprocessing (remove stopwords&special chars; tokenize) ▪ Convert tweets into vectors ▪ Train classifiers with 10-fold cross-validation ▪ Evaluate results as usual (precision, recall, F-Measure) ▪ Results ranging from 0.38 to 0.76 in F-Measure
  57. 57. 57 1. Definition of metrics ⚫ In the basis of: ◼ which are the business priorities of the project ◼ what data is available (i.e., can be effectively used) CIbSE-RET2019©XavierFranch
  58. 58. 58 Advanced feedback gathering forms CIbSE-RET2019©XavierFranch
  59. 59. 59 Motivating users to give feedback • Motivation ▪ Extrinsic (incentives) ▪ Gamification ▪ Persuasive technologies ▪ Social networks ▪ Adapted to every type of user CIbSE-RET2019©XavierFranch
  60. 60. 60 Reliability • The problem of fake reviews ▪ Including bots ▪ It has an industry behind! Preprint! CIbSE-RET2019©XavierFranch
  61. 61. 61 Example Speech-actt analysis Opinions & discussions Annotated sentences CIbSE-RET2019©XavierFranch
  62. 62. 62 Complex ecosystem CIbSE-RET2019©XavierFranch
  63. 63. 63 Data Integration ontology CIbSE-RET2019©XavierFranch

×