The document discusses data-driven requirements engineering (RE), which uses data collected from system usage to inform the RE process. It describes traditional RE and how data-driven RE aims to supplement it using explicit feedback from users and implicit feedback from usage data. The data is analyzed using natural language processing, machine learning, and topic modeling to categorize issues and identify topics. Challenges of data-driven RE include integrating it into development processes, collecting and processing heterogeneous data, and gaining user trust. Overall, it presents an opportunity to deliver more business value but requires the right approach and consideration of traditional RE methods.

1. Data-Driven Requirements
Engineering: Principles, Methods
and Challenges
Xavier Franch
Universitat Politècnica de Catalunya (UPC-BarcelonaTech)
Tutorial at RCIS’20, September
(virtually) Limassol, Cyprus

2. 2
RE (still) central in SERCIS2020XavierFranch

3. 3
RE (still) a problem in SE
Causes of failure
RCIS2020XavierFranch

4. 4
The question then is...
How to ensure that a system is delivering
the right value to its stakeholders?
RCIS2020XavierFranch

5. 5
From traditional RE...RCIS2020XavierFranch

6. 6
...to data-driven RERCIS2020XavierFranch

7. 7
Data-driven RE
“requirements engineering by the masses and for the masses”
RCIS2020XavierFranch

8. 8
RCIS2020XavierFranch

9. 9
A word of cautionRCIS2020XavierFranch

10. 10
The data-driven RE cycleRCIS2020XavierFranch
Feedback gathering
and analysis
Decision making

11. 11
EXPLICIT FEEDBACK
RCIS2020XavierFranch

12. 12
Explicit Feedback
• Feedback directly provided by the user
• Main phases:
gathering analysing acting upon
RCIS2020XavierFranch

13. 13
Gathering explicit feedbackRCIS2020XavierFranch
style mode
advanced
features
channel

14. 14
Explicit feedback analysis
• Preprocessing
• Categorization
• Sentiment analysis
• Topic modeling
RCIS2020XavierFranch

15. 15
Behind the curtains
• NLP: processing textual information
• ML: learning from data
RCIS2020XavierFranch

16. 16
Analysis: Preprocessing
• From text to lexical/syntactical units
 Tokenization
 Stemming / Lemmatization
 Part-of-speech tagging
“I have a problem when saving the document, please check it”
I/PRP have/VBP a/DT problem/NN when/WRB saving/VBG
the/DT document/NN ,/, please/VBP check/VB it/PRP
RCIS2020XavierFranch

17. 17
Analysis: Categorization
• Classical: Bug report or feature request
Speech-actt
analysis
Opinions &
discussions
Annotated
sentences
“I have a problem when saving
the document, please check it”
RCIS2020XavierFranch

18. 18
Can be more elaboratedRCIS2020XavierFranch

19. 19
Analysis: Sentiment analysis
Deciding if a piece of text expresses a particular affect or mood
But of course, not easy...:
“Great, I like this feature that gives me this lovely headache”
had fun using it before but now it is really horrible :( help!!
+2 -2-1 -1 -1[+2,-5]
uploading pictures with the app is so annoying!
pleeeeeeease add an unlike button and I will love you forever!!
[+1,-3]
[+5,-1]
Approaches: use of dictionaries and ML/DL techniques
RCIS2020XavierFranch

20. 20
Analysis: Topic modeling
• Identifying topics that best describes a corpus
 Each topic described by a (probabilistic) distribution of words
• Most popular algorithm: Latent Dirichlet Allocation
• Challenge: parameterization
RCIS2020XavierFranch

21. 21
Putting all pieces togetherRCIS2020XavierFranch

22. 22
Putting all pieces togetherRCIS2020XavierFranch

23. 23
State of the art in NLP
https://www.slideshare.net/alessio_ferrari/requirements-
engineering-focus-on-natural-language-processing-lecture-2
RCIS2020XavierFranch

24. 24
Challenges with explicit feedbackRCIS2020XavierFranch

25. 25
IMPLICIT FEEDBACK
RCIS2020XavierFranch

26. 26
Monitoring QoSRCIS2020XavierFranch

27. 27
Usage data
• Information about usage of the system
• What can be discovered?
 Which functionalities are most used
 Which navigational paths prevail (or not)
 Which calls result often in error codes
 …
 Especially if QoS also available
RCIS2020XavierFranch

28. 28
Challenges with usage logs
• Data noise
• Incomplete data
• Concept of session
• Evolution
• Anonymization
• …
RCIS2020XavierFranch

29. 29
Boosting in IoT and mobile devicesRCIS2020XavierFranch

30. 30
Importance of contextRCIS2020XavierFranch

31. 31
Contextual requirements
(context, behavior)
Drowsy driver Ring alarm
Satisfaction: given cr = (ctx, beh)
eval(cr) = ctx  dom(Oper)  evaluable(ctx) 
ctx  beh
Objective function:
minimize {cri: 0 < i ≤ m: NOT EvalCR(cri)}
RCIS2020XavierFranch

32. 32
Feedback channel combinationRCIS2020XavierFranch
• Bringing the best of several worlds

33. 33
Collection, processing, and integration of
relevant heterogeneous information
RCIS2020XavierFranch Feedback channel combination

34. 34
Crowd-based RERCIS2020XavierFranch

35. 35
DECISION MAKING
RCIS2020XavierFranch

36. 36
Keys to decision making
software analytics
release planning
RCIS2020XavierFranch

37. 37
Code analytic tools
Valuable, but not strategic enough
RCIS2020XavierFranch

38. 38
Software analytics toolsRCIS2020XavierFranch

39. 39
Example: Q-Rapids dashboardRCIS2020XavierFranch

40. 40
Techniques
prediction
what-if
analysis
RCIS2020XavierFranch

41. 41
Elaborating indicatorsRCIS2020XavierFranch

42. 42
Elicitation of requirements

43. 43
Release planning
• Closing the data-driven RE cycle
RCIS2020XavierFranch

44. 44
Release Planning in DDRERCIS2020XavierFranch

45. 45
Involving relevant stakeholdersRCIS2020XavierFranch

46. 46
Liquid democracy
• Stakeholders can evaluate
requirements based on
different interest dimensions
 Most stakeholders might not be
appropriate to evaluate all
requirements accurately
 Stakeholders can selectively
delegate votes to experts
RCIS2020XavierFranch

47. 47
THE WAY AHEAD
RCIS2020XavierFranch

48. 48
The way ahead
• Six research challenges for data-driven RE
RCIS2020XavierFranch

49. 49
Challenge 1
Seamless integration into existing development
processes and software systems
RCIS2020XavierFranch

50. 50
Challenge 1
Seamless integration into existing development
processes and software systems
RCIS2020XavierFranch

51. 51
Challenge 2
Collection, processing, and integration of
relevant heterogeneous information
RCIS2020XavierFranch

52. 52
Several variations existRCIS2020XavierFranch

53. 53
Challenge 3
Context-awareness and adaptability
RCIS2020XavierFranch

54. 54
Self-adapting monitoringRCIS2020XavierFranch

55. 55
Challenge 4
Provision of actionable feedback
RCIS2020XavierFranch

56. 56
Challenge 5
Gaining users’ trust
RCIS2020XavierFranch

57. 57
Challenge 6
Provision of value for the entire life-cycle
RCIS2020XavierFranch

58. 58
Challenges to adoption
• Organizational
 Tailoring to the company
 Integration with company WoW
 Shared vocabulary
• Value
 Informativeness
 Transparency
• Technological
 Simplify tool installation
 Efficient tool configuration
RCIS2020XavierFranch

59. 59
Challenges to adoptionRCIS2020XavierFranch

60. 60
Lessons learned (I)
• Not for free
 Adapt process and mindset
 Develop infrastructure
• Different for every company
 Processes need to be customizable
• Expertise needed
 Specialized roles (e.g. data scientists)
RCIS2020XavierFranch

61. 61
Lessons learned (II)
• Needs to be implemented incrementally
 Gradually master intricacies
 Create awareness in the organization
• Transparency as a must
 Clear rationale behind decisions
 Shown to relevant stakeholders
RCIS2020XavierFranch

62. 62
CONCLUSIONS
RCIS2020XavierFranch

63. 63
Summary

64. 64
Final considerations
• Offers a great opportunity for delivering more
business value to systems’ stakeholders
• Some considerations
 Not a hammer for every nail
 Data-driven  needs data
• Still traditional methods at least to start with
• The role of traditional RE in the loop is worth considering
 New, but maybe not as much
• E.g., compare to online experimentation
– A/B tests, fake door tests, …
• Other areas: UCD, process mining, MSR, …
RCIS2020XavierFranch

65. 65
Thanks to...
• Anna Perini, Fabiano Dalpiaz, Marc Oriol,
Silverio Martínez for comments on early
versions and associated documents
RCIS2020XavierFranch
Summary at:
Franch, X.: Data-Driven Requirements Engineering: Principles, Methods
and Challenges. In Proceedings of the 14th International Conference on
Research Challenges in Information Science (RCIS 2020). LNBIP 385:
625-626, Springer, 2020.
Available at:
https://www.researchgate.net/publication/344328110_Data_Driven_Requi
rements_Engineering_Principles_Methods_and_Challenges

66. 66
Images All other images labelled for reuse

67. Data-Driven Requirements
Engineering: Principles, Methods
and Challenges
Xavier Franch
Universitat Politècnica de Catalunya (UPC-BarcelonaTech)
Tutorial at RCIS’20, September
(virtually) Limassol, Cyprus