The document discusses new opportunities for using natural language processing (NLP) techniques to support requirements conversations between analysts and stakeholders. It presents an initial tool called Trace2Conv that aims to trace formal requirements back to informal conversations to provide additional context. A second tool called Requirements Conversation Summarizer is proposed to facilitate analysts in exploring long recorded conversation transcripts by identifying questions and filtering information by relevance.

1. Requirements Conversations:
A New Frontier in AI-for-RE
Fabiano Dalpiaz
Requirements Engineering Lab
Utrecht University, the Netherlands
August 16, 2022
f.dalpiaz@uu.nl @FabianoDalpiaz

2. Outline and Acks
@2022 Fabiano Dalpiaz
2
1. Context:
NLP4RE
2. Conversational
Req. Engineering
3. From reqs to
specs
4. Conversation
summarization
5. Outlook

3. Outline and Acks
@2022 Fabiano Dalpiaz
2
1. Context:
NLP4RE
2. Conversational
Req. Engineering
3. From reqs to
specs
4. Conversation
summarization
5. Outlook
Sjaak Brinkkemper
Tjerk Spijkman
Nikita van den Berg
Xavier de Bondt

4. 1. Context: NLP for Requirements
Engineering (NLP4RE)
Fabiano Dalpiaz,Alessio Ferrari, Xavier Franch, and Cristina Palomares. "Natural language processing
for requirements engineering:The best is yet to come." IEEE Software 35, no. 5 (2018): 115-119.
@2022 Fabiano Dalpiaz
3

5. RE practice: most reqs. are in natural language
@2022 Fabiano Dalpiaz
4
The <system name> shall <system response>.
WHILE <in a specific state> the <system name> shall <system
response>
WHEN <trigger> the <system name> shall <system response>
…
User stories
App store
reviews
EARS
Rupp’s template & ISO/IEC/IEEE 29148
Use cases

6. RE Research: 4 categories of NLP4RE tools
@2022 Fabiano Dalpiaz
5
1. Find defects /
deviations from
good practice
2. Generate models
from NL requirements
3. Infer trace links
between NL
requirements and
other artifacts
4. Identify key
abstractions
from NL
documents
Daniel Berry, Ricardo Gacitua, Pete Sawyer, and Sri Fatimah Tjong. "The case for dumb
requirements engineering tools." In InternationalWorking Conference on Requirements
Engineering: Foundation for Software Quality, pp. 211-217. 2012.

7. An active area of research!
@2022 Fabiano Dalpiaz
6
Liping Zhao,Waad Alhoshan, Alessio Ferrari,
Keletso J. Letsholo, Muideen A.Ajagbe, Erol-
Valeriu Chioasca, and Riza T. Batista-Navarro.
"Natural Language Processing (NLP) for
Requirements Engineering:A Systematic
Mapping Study." ACM Computing Surveys 54:3,
2022

8. An active area of research!
@2022 Fabiano Dalpiaz
6
Liping Zhao,Waad Alhoshan, Alessio Ferrari,
Keletso J. Letsholo, Muideen A.Ajagbe, Erol-
Valeriu Chioasca, and Riza T. Batista-Navarro.
"Natural Language Processing (NLP) for
Requirements Engineering:A Systematic
Mapping Study." ACM Computing Surveys 54:3,
2022
Specification
Main artifact for
AI-based tools

9. Elicitation: the root of (all) NL requirements
@2022 Fabiano Dalpiaz
7
Specification

10. Elicitation: the root of (all) NL requirements
@2022 Fabiano Dalpiaz
7
Requirements
conversations
Requirements Analyst
Own ideas
Budget / project
constraints
Design
decisions Domain-specific
documentation
Specification

11. Elicitation: the root of (all) NL requirements
@2022 Fabiano Dalpiaz
7
Requirements
conversations
Requirements Analyst
Own ideas
Budget / project
constraints
Design
decisions Domain-specific
documentation
Specification

12. Elicitation is heavily centered on conversations!
@2022 Fabiano Dalpiaz
8
Requirements
conversations
Requirements Analyst
Own ideas
Budget / project
constraints
Design
decisions Domain-specific
documentation

13. Elicitation is heavily centered on conversations!
@2022 Fabiano Dalpiaz
8
Requirements
conversations
Requirements Analyst
Own ideas
Budget / project
constraints
Design
decisions Domain-specific
documentation
NaPiRE (August 8, 2022)
http://www.re-survey.org/#/explore
˝

14. Timeliness: why researching conversations now?
@2022 Fabiano Dalpiaz
9

15. Timeliness: why researching conversations now?
@2022 Fabiano Dalpiaz
9
Increased remote work
and collaboration

16. Timeliness: why researching conversations now?
@2022 Fabiano Dalpiaz
9
Increased remote work
and collaboration
Automated
transcription

17. 2. Conversational RE
@2022 Fabiano Dalpiaz
10

18. Background theory: Refinement in RE
@2022 Fabiano Dalpiaz
11
Pohl, Klaus. "The three dimensions of requirements
engineering: a framework and its applications."
Information systems 19.3 (1994): 243-258.
Specification
Representation
opaque
fair
complete
common view
informal semi-formal formal
personal view
Initial RE
input
Desired RE output
Agreement

19. Background theory: Refinement in RE
@2022 Fabiano Dalpiaz
11
Pohl, Klaus. "The three dimensions of requirements
engineering: a framework and its applications."
Information systems 19.3 (1994): 243-258.
Specification
Representation
opaque
fair
complete
common view
informal semi-formal formal
personal view
Initial RE
input
Desired RE output
Agreement
Refinement
path in
practice

20. Background theory: Refinement in RE
@2022 Fabiano Dalpiaz
11
Pohl, Klaus. "The three dimensions of requirements
engineering: a framework and its applications."
Information systems 19.3 (1994): 243-258.
Specification
Representation
opaque
fair
complete
common view
informal semi-formal formal
personal view
Initial RE
input
Desired RE output
Agreement
Refinement
path in
practice
RE research approaches,
including NLP4RE Tools

21. How do current NLP4RE tools work?
@2022 Fabiano Dalpiaz
12

22. NLP4RE Tools for Conversational RE
@2022 Fabiano Dalpiaz
13
vs.

23. NLP4RE Tools for Conversational RE
@2022 Fabiano Dalpiaz
13
} The tool supports the conversation between analyst and stakeholders
vs.

24. NLP4RE Tools for Conversational RE
@2022 Fabiano Dalpiaz
13
} The tool supports the conversation between analyst and stakeholders
} Elicitation and refinement as concurrent activities
vs.

25. Conversational RE
@2022 Fabiano Dalpiaz
14
“The (automated) analysis of requirements
elicitation conversations aimed at identifying and
extracting requirements-relevant information”

26. (Requirements) conversations vs. specifications
@2022 Fabiano Dalpiaz
15

27. (Requirements) conversations vs. specifications
@2022 Fabiano Dalpiaz
15
2+ parties (here Analyst
and Stakeholder)

28. (Requirements) conversations vs. specifications
@2022 Fabiano Dalpiaz
15
2+ parties (here Analyst
and Stakeholder)
Informal: no “shall”
statements, user
stories, glossary

29. (Requirements) conversations vs. specifications
@2022 Fabiano Dalpiaz
15
2+ parties (here Analyst
and Stakeholder)
Informal: no “shall”
statements, user
stories, glossary
Relevant
information may
be sparse

30. (Requirements) conversations vs. specifications
@2022 Fabiano Dalpiaz
15
2+ parties (here Analyst
and Stakeholder)
Informal: no “shall”
statements, user
stories, glossary
Relevant
information may
be sparse
Includes persuasion,
uncertainty,
misunderstandings

31. Dissecting a conversation: turns and grounding acts
@2022 Fabiano Dalpiaz
16
Traum, David R., and Elizabeth A. Hinkelman.
"Conversation acts in task-oriented spoken dialogue."
Computational intelligence 8.3 (1992): 575-599.

32. Dissecting a conversation: turns and grounding acts
@2022 Fabiano Dalpiaz
16
Traum, David R., and Elizabeth A. Hinkelman.
"Conversation acts in task-oriented spoken dialogue."
Computational intelligence 8.3 (1992): 575-599.
Turns and utterance units as
atomic entities

33. Dissecting a conversation: turns and grounding acts
@2022 Fabiano Dalpiaz
16
Traum, David R., and Elizabeth A. Hinkelman.
"Conversation acts in task-oriented spoken dialogue."
Computational intelligence 8.3 (1992): 575-599.
Turns and utterance units as
atomic entities
Grounding acts determine
the effect of an
utterance unit

34. Dissecting a conversation: discourse units
@2022 Fabiano Dalpiaz
17
Traum, David R., and Elizabeth A. Hinkelman.
"Conversation acts in task-oriented spoken dialogue."
Computational intelligence 8.3 (1992): 575-599.

35. Dissecting a conversation: discourse units
@2022 Fabiano Dalpiaz
17
Cross-speaker interaction
defines the meaning
Traum, David R., and Elizabeth A. Hinkelman.
"Conversation acts in task-oriented spoken dialogue."
Computational intelligence 8.3 (1992): 575-599.

36. Traum, David R., and Elizabeth A. Hinkelman.
"Conversation acts in task-oriented spoken dialogue."
Computational intelligence 8.3 (1992): 575-599.
Dissecting a conversation: argumentation acts
@2022 Fabiano Dalpiaz
18

37. Traum, David R., and Elizabeth A. Hinkelman.
"Conversation acts in task-oriented spoken dialogue."
Computational intelligence 8.3 (1992): 575-599.
Dissecting a conversation: argumentation acts
@2022 Fabiano Dalpiaz
18
The purpose of a
conversation across
multiple turns:
argumentation acts

38. Traum, David R., and Elizabeth A. Hinkelman.
"Conversation acts in task-oriented spoken dialogue."
Computational intelligence 8.3 (1992): 575-599.
Dissecting a conversation: argumentation acts
@2022 Fabiano Dalpiaz
18
The purpose of a
conversation across
multiple turns:
argumentation acts
Q&A as a basic interaction,
clarifications, summary,
persuasion, …

39. Conversations vs. Specifications: not quite the same
@2022 Fabiano Dalpiaz
19
The parameters A and B shall be configured via
a configuration file in format XYZ
[specification]
[conversation]

40. Tools for Conversational RE: Two Examples
@2022 Fabiano Dalpiaz
20
Trace2Conv:
pre-RS traceability
Requirements Conversation
Summarizer

41. 3. Trace2Conv: Tracing
requirements to conversations
@2022 Fabiano Dalpiaz
21

42. 3. Trace2Conv: Tracing
requirements to conversations
@2022 Fabiano Dalpiaz
21

43. Requirements
conversations
Requirements Analyst
Own ideas
Budget / project
constraints
Design
decisions Domain-specific
documentation
Trace2Conv: Key Idea
@2022 Fabiano Dalpiaz
22
Specification

44. Requirements
conversations
Requirements Analyst
Own ideas
Budget / project
constraints
Design
decisions Domain-specific
documentation
Trace2Conv: Key Idea
@2022 Fabiano Dalpiaz
22
} Supports backward, pre-RS traceability
} Largely overlooked area of research
Specification
Trace2Conv

45. Requirements
conversations
Requirements Analyst
Own ideas
Budget / project
constraints
Design
decisions Domain-specific
documentation
Trace2Conv: Key Idea
@2022 Fabiano Dalpiaz
22
} Supports backward, pre-RS traceability
} Largely overlooked area of research
} Aims to find information that provides
additional context to a requirement
Specification
Trace2Conv

46. Requirements
conversations
Requirements Analyst
Own ideas
Budget / project
constraints
Design
decisions Domain-specific
documentation
Trace2Conv: Key Idea
@2022 Fabiano Dalpiaz
22
} Supports backward, pre-RS traceability
} Largely overlooked area of research
} Aims to find information that provides
additional context to a requirement
} Has to cope with an abstraction gap
} Formal to informal
Specification
Trace2Conv

47. What can we achieve with Trace2Conv?
@2022 Fabiano Dalpiaz
23
Trace requirements back to elicitation sessions

48. What can we achieve with Trace2Conv?
@2022 Fabiano Dalpiaz
23
Trace requirements back to elicitation sessions
Obtain client agreement by showing requirement sources

49. What can we achieve with Trace2Conv?
@2022 Fabiano Dalpiaz
23
Trace requirements back to elicitation sessions
Obtain client agreement by showing requirement sources
Enable iterative specification writing and review

50. What can we achieve with Trace2Conv?
@2022 Fabiano Dalpiaz
23
Trace requirements back to elicitation sessions
Obtain client agreement by showing requirement sources
Enable iterative specification writing and review
Provide extra context about a requirement for devs

51. Architectural Design – Inputs and Backend
@2022 Fabiano Dalpiaz
24

52. Architectural Design – Inputs and Backend
@2022 Fabiano Dalpiaz
24

53. Pre-processing and matching
@2022 Fabiano Dalpiaz
25
As a vendor user, I can use the password forgotten
functionality whenever I forgot or want to reset my
password, so that I always have a way to create a new
password

54. Pre-processing and matching
@2022 Fabiano Dalpiaz
25
As a vendor user, I can use the password forgotten
functionality whenever I forgot or want to reset my
password, so that I always have a way to create a new
password

55. Pre-processing and matching
@2022 Fabiano Dalpiaz
25
As a vendor user, I can use the password forgotten
functionality whenever I forgot or want to reset my
password, so that I always have a way to create a new
password

56. Pre-processing and matching
@2022 Fabiano Dalpiaz
25
As a vendor user, I can use the password forgotten
functionality whenever I forgot or want to reset my
password, so that I always have a way to create a new
password

57. Architectural Design - Frontend
@2022 Fabiano Dalpiaz
26

58. Short demo of the Trace2Conv frontend
@2022 Fabiano Dalpiaz
27

59. Short demo of the Trace2Conv frontend
@2022 Fabiano Dalpiaz
27

60. Trace2Conv: Next Steps
@2022 Fabiano Dalpiaz
28
Requirements evolution over
multiple conversations
More advanced heuristics: what
are the most likely matches?
Matching segments rather
than speaker turns

61. 4. Requirements Conversations
Summarizer
@2022 Fabiano Dalpiaz
29

62. } Aim: summarization before a specification exists
} Trigger: long recorded conversations, spanning over multiple hours
} How to facilitate the analyst in exploring the transcript?
Summarizing a transcript: ideas
@2022 Fabiano Dalpiaz
30

63. } Aim: summarization before a specification exists
} Trigger: long recorded conversations, spanning over multiple hours
} How to facilitate the analyst in exploring the transcript?
Summarizing a transcript: ideas
@2022 Fabiano Dalpiaz
30
Idea #1: Identify
the questions

64. } Aim: summarization before a specification exists
} Trigger: long recorded conversations, spanning over multiple hours
} How to facilitate the analyst in exploring the transcript?
Summarizing a transcript: ideas
@2022 Fabiano Dalpiaz
30
Idea #1: Identify
the questions
Idea #2: Filter by
question relevance

65. } Aim: summarization before a specification exists
} Trigger: long recorded conversations, spanning over multiple hours
} How to facilitate the analyst in exploring the transcript?
Summarizing a transcript: ideas
@2022 Fabiano Dalpiaz
30
Idea #1: Identify
the questions
Idea #2: Filter by
question relevance
Idea #3: Label by
relevance type [WiP]

66. How to identify the questions? (Idea #1)
@2022 Fabiano Dalpiaz
31

67. How to identify the questions? (Idea #1)
@2022 Fabiano Dalpiaz
31
Based on sequences of POS tags:
Wh-, yes/no, tag questions

68. How to identify the questions? (Idea #1)
@2022 Fabiano Dalpiaz
31
Based on sequences of POS tags:
Wh-, yes/no, tag questions
Based on pre-trained DistilBert:
28/38 question tags matched

69. How to identify the questions? (Idea #1)
@2022 Fabiano Dalpiaz
31
Based on sequences of POS tags:
Wh-, yes/no, tag questions
Based on pre-trained DistilBert:
28/38 question tags matched
Combination: question if either
approach says so

70. Is Idea #1 effective?
@2022 Fabiano Dalpiaz
32
334 30
74 736
254 110
74 736
349 15
105 705

71. How to filter relevant questions? (Idea #2, version A)
@2022 Fabiano Dalpiaz
33

72. How to filter relevant questions? (Idea #2, version A)
@2022 Fabiano Dalpiaz
33
TF-IDF can be used to distinguish
questions with domain-specific
words from general ones

73. How to filter relevant questions? (Idea #2, version B)
@2022 Fabiano Dalpiaz
34

74. How to filter relevant questions? (Idea #2, version B)
@2022 Fabiano Dalpiaz
34
A context document can be, e.g., a
system/project definition document

75. What is our gold standard for relevance?
@2022 Fabiano Dalpiaz
35

76. What is our gold standard for relevance?
@2022 Fabiano Dalpiaz
35

77. What is our gold standard for relevance?
@2022 Fabiano Dalpiaz
35

78. Is Idea #2 effective?
@2022 Fabiano Dalpiaz
36
No large differences between the
approaches – Ideas #2A and #2B
are practically equivalent

79. Summarization: outlook
@2022 Fabiano Dalpiaz
37
} What does relevance mean?
} Large disagreement, especially on
questions

80. Summarization: outlook
@2022 Fabiano Dalpiaz
37
} What does relevance mean?
} Large disagreement, especially on
questions
} How much can we summarize?
} End goal of the tool

81. 5. Outlook
@2022 Fabiano Dalpiaz
38

82. Tools for Conversational RE: Two Examples
@2022 Fabiano Dalpiaz
39
Trace2Conv:
pre-RS traceability
Requirements Conversation
Summarizer

83. Direction #3: distilling requirements?
@2022 Fabiano Dalpiaz
40
} Can we automatically generate
requirements from conversations?
} Long-term direction
} High value
} Extremely challenging
} Rarely mentioned in an explicit way
(Spijkman, CAiSE’21)

84. Direction #3: distilling requirements?
@2022 Fabiano Dalpiaz
40
Spijkman & al.,
NLP4RE’21
Ruiz & Hasselman,
EMMSAD’20
} Can we automatically generate
requirements from conversations?
} Long-term direction
} High value
} Extremely challenging
} Rarely mentioned in an explicit way
(Spijkman, CAiSE’21)

85. The future of evaluation metrics
@2022 Fabiano Dalpiaz
41
} Most of the literature employs information retrieval metrics
} Precision, recall, F1, …
} Progressive shift toward quality-in-use with conversational RE tools?

86. The future of evaluation metrics
@2022 Fabiano Dalpiaz
41
} Most of the literature employs information retrieval metrics
} Precision, recall, F1, …
} Progressive shift toward quality-in-use with conversational RE tools?
ISO 25022
(quality in use)

87. The way ahead…
Today’s NLP4RE Tools
@2022 Fabiano Dalpiaz
42

88. The way ahead…
Today’s NLP4RE Tools
Conversational RETools
@2022 Fabiano Dalpiaz
42

89. Thank you for listening! Questions?
f.dalpiaz@uu.nl @FabianoDalpiaz
RE-Lab’s research illustrated, 2018