This document presents an approach for change impact analysis of natural language requirements. It aims to identify requirements that may be impacted by a change to reduce expensive manual analysis. The approach uses natural language processing techniques to detect phrases and calculate similarity between requirements. Requirements are then sorted based on similarity to the changed requirement. Propagation conditions expressed as Boolean queries help identify impacted requirements by traversing the sorted list. An evaluation of the approach on two case studies found it was effective, with minimal futile inspection effort, and recommended best similarity measures for each case. Future work includes addressing tacit dependencies and more empirical studies.

1. .lusoftware verification & validation
VVS
Change Impact Analysis for
Natural Language Requirements:
Chetan Arora, Mehrdad Sabetzadeh,
Arda Goknil, Lionel Briand
Frank Zimmer
University of Luxembourg,
Luxembourg
SES TechCom,
Luxembourg
An NLP Approach

2. Problem Definition
• Requirements change frequently
• Large number of requirements and interdependencies
• Consistency must be maintained
• Handling change is expensive
• Support is required for impact analysis among requirements
2

3. 3
Scope
Requirements
Architecture
& Design Source Code
Test
Cases
Software
Documentation
Majority of Research

4. Challenges

5. Example
• R1: The mission operation controller shall transmit satellite
status reports to the user help desk.
• R2: The satellite management system shall provide users with
the ability to transfer maintenance and service plans to the
user help desk.
• R3: The mission operation controller shall transmit any
detected anomalies to the user help desk.
5

6. What Changed?
• R1: The mission operation controller shall transmit satellite
status reports to the user help desk document repository.
• R2: The satellite management system shall provide users with
the ability to transfer maintenance and service plans to the
user help desk.
• R3: The mission operation controller shall transmit any
detected anomalies to the user help desk.
6
user help desk? operator help desk?
station maintenance crew help desk?

7. What Changed?
• R1: The mission operation controller shall transmit satellite
status reports to the user help desk document repository.
• R2: The satellite management system shall provide users with
the ability to transfer maintenance and service plans to the
user help desk.
• R3: The mission operation controller shall transmit any
detected anomalies to the user help desk.
7
user help desk

8. Why Was the Change Made?
• R1: The mission operation controller shall transmit satellite
status reports to the user help desk document repository.
• R2: The satellite management system shall provide users with
the ability to transfer maintenance and service plans to the
user help desk.
• R3: The mission operation controller shall transmit any
detected anomalies to the user help desk.
8
Possible Reason : Replace (user help desk) with
(user document repository)

9. Why Was the Change Made?
• R1: The mission operation controller shall transmit satellite
status reports to the user help desk document repository.
• R2: The satellite management system shall provide users with
the ability to transfer maintenance and service plans to the
user help desk.
• R3: The mission operation controller shall transmit any
detected anomalies to the user help desk.
9
Another Possible Reason : No communication between
(user help desk) and
(mission operation controller)

10. Approach

11. Approach
11
Apply
Change
Identify
Differences
Specify
Propagation
Condition Propagation
Condition
Process
Requirements
2
6
6
6
6
6
6
4
s11 ··· s1n
... ... ...
sn1 ··· snn
3
7
7
7
7
7
7
5
Phrases
Similarity
Matrix
Requirements
Document
Sort
Requirements
Sorted
Requirements
Syntactic Similarity:
satellite transmission licence
license of satellite transmission
Semantic Similarity:
transmit
transfer
Propagation Condition as a Boolean Query:
(user help desk)
AND
(mission operation controller)
AND
(transmit)

12. Approach
Apply
Change
Identify
Differences
Specify
Propagation
Condition Propagation
Condition
Process
Requirements
2
6
6
6
6
6
6
4
s11 ··· s1n
... ... ...
sn1 ··· snn
3
7
7
7
7
7
7
5
Phrases
Similarity
Matrix
Requirements
Document
Sort
Requirements
Sorted
Requirements

13. Processing Requirements
Statements
• R1: The mission operation controller shall transmit satellite
status reports to the user help desk.
• R2: The satellite management system shall provide users with
the ability to transfer maintenance and service plans to the
user help desk.
• R3: The mission operation controller shall transmit any
detected anomalies to the user help desk.
13

14. Processing Requirements
Statements
• R1: The mission operation controller shall transmit satellite
status reports to the user help desk.
• R2: The satellite management system shall provide users with
the ability to transfer maintenance and service plans to the
user help desk.
• R3: The mission operation controller shall transmit any
detected anomalies to the user help desk.
14
Phrase Detection Similarity Calculation
1.0
transmit
transfer
Noun Phrase (NP)
Verb Phrase (VP)

15. Approach
Apply
Change
Identify
Differences
Specify
Propagation
Condition Propagation
Condition
Process
Requirements
2
6
6
6
6
6
6
4
s11 ··· s1n
... ... ...
sn1 ··· snn
3
7
7
7
7
7
7
5
Phrases
Similarity
Matrix
Requirements
Document
Sort
Requirements
Sorted
Requirements

16. 16
NAtural language Requirements Change Impact Analyzer
https://sites.google.com/site/svvnarcia/
Demo

18. monitoring desk
user help desk
Requirement Phrases
Impact Likelihood Computation
18
user priority list of his help desk
Propagation
Condition

19. 19
Sorted Requirements
Using the Sorted List

20. Inspect till the point after which the quantitative measure loses the
capability to sufficiently differentiate the impacted requirements
20
When to Stop Inspecting?
Graph Builder
Membership & Difference vs. %Inspected
%Inspected
0,00 0,05 0,10 0,15 0,20 0,25 0,30 0,35 0,40 0,45 0,50 0,55 0,60 0,65 0,70 0,75 0,80 0,85 0,90 0,95 1,00
0,00
0,20
0,40
0,60
0,80
0,00
0,10
0,20
0,30
Where(40160 rows excluded)
h
Delta
% of requirements traversed in the sorted list
h
0 20 40 60 80 100
0.0
0.1
0.2
0.3
0.0
1.0
0.8
0.6
0.4
0.2
Matchingscore
max
h /3max last
Graph Builder
Membership & Difference vs. %Inspected
%Inspected
0,00 0,05 0,10 0,15 0,20 0,25 0,30 0,35 0,40 0,45 0,50 0,55 0,60 0,65 0,70 0,75 0,80 0,85 0,90 0,95 1,00
Membership
0,00
0,20
0,40
0,60
0,80
Difference
0,00
0,10
0,20
0,30
Where(40160 rows excluded)
h
Delta
% of requirements traversed in the sorted list
h
0 20 40 60 80 100
0.0
0.1
0.2
0.3
0.0
1.0
0.8
0.6
0.4
0.2
Matchingscore
max
h /3max last
ImpactLikelihood

21. Inspect till the point after which the quantitative measure loses the
capability to sufficiently differentiate the impacted requirements
ImpactLikelihoodDelta
When to Stop Inspecting?
Graph Builder
Membership & Difference vs. %Inspected
%Inspected
0,00 0,05 0,10 0,15 0,20 0,25 0,30 0,35 0,40 0,45 0,50 0,55 0,60 0,65 0,70 0,75 0,80 0,85 0,90 0,95 1,00
Membership
0,00
0,20
0,40
0,60
0,80
Difference
0,00
0,10
0,20
0,30
Where(40160 rows excluded)
h
Delta
% of requirements traversed in the sorted list
h
0 20 40 60 80 100
0.0
0.1
0.2
0.3
0.0
1.0
0.8
0.6
0.4
0.2
Matchingscore
max
h /3max last
Recommended point in sorted requirements to stop inspecting

22. Empirical Evaluation

23. Research Questions
• Which similarity measures perform best?
• How effective is our approach, using the guidelines?
23

24. Case Studies
24
Case-A
• 160 Requirements
• 9 Change Scenarios
Case-B• 72 Requirements
• 5 Change Scenarios

25. Change Scenarios
25
ID Propagation Condition Pattern Size of Impact
SetA.1 ⟨NP⟩ AND ⟨NP⟩ 4
A.2 ⟨NP⟩ OR ⟨NP⟩ 8
A.3 ⟨NP⟩ 39
A.4 (⟨NP⟩ OR ⟨NP⟩) AND ⟨NP⟩ 5
A.5 ⟨NP⟩ OR ⟨NP⟩ 10
A.6 ⟨NP⟩ AND ⟨NP⟩ 3
A.7 ⟨NP⟩ AND ⟨NP⟩ 7
A.8 ⟨NP⟩ OR ⟨NP⟩ 5
A.9 ⟨verbatim-text⟩ AND ⟨NP⟩ 3
B.1 ⟨NP⟩ AND ⟨NP⟩ 2
B.2 ⟨NP⟩ 9
B.3 ⟨NP⟩ AND ⟨NP⟩AND ⟨NP⟩ 1
B.4 ⟨NP⟩ AND ⟨NP⟩ 1
B.5 (⟨NP⟩ OR ⟨NP⟩) AND (⟨NP⟩ OR ⟨NP⟩) 9

26. 26
Syntactic Measures
Block Distance
Cosine Similarity
Dice’s coefficient
Euclidean
Jaccard
Jaro
Jaro Winkler
Levenstein
Monge Elkan
SOFTTFIDF
Semantic Measures
HSO
JCN
LCH
LESK
LESK_TANIM
LIN
PATH
RES
WUP
Which Similarity Measures
Perform Best?
Recommended
Best in
Case-A Best in
Case-B

27. “touristic attraction”
is a
“point of interest”
Reason:
Lack of a Domain Model
1 impacted requirement missed
out of a total of 106 impacted
requirements.
Effectiveness of Our Approach
27
FutileInspectionEffort
1% - 7% 6% - 8%
45%

28. Key Points from Evaluation
28
Choice of
Similarity Measures
Effectiveness Execution Time

29. Related Work
• A. Goknil et. al. 2014 - Change Impact Analysis in requirements using dependency
model with formal semantics
• J. Cleland-Huang et. al. (2005) - Soft goal dependencies for analysing the impact
of changes in functional requirements to non-functional requirements
• Yang et. al. (2011) - Use of NLP (text chunking) for resolving ambiguities in
requirements
• J. Cleland-Huang, “Traceability in agile projects,” in Software and Systems
Traceability, J. Cleland-Huang, O. Gotel, and A. Zisman, Eds. Springer, 2012.
29
Just-In-Time Traceability

30. Future Work
• Address the limitation concerning tacit dependencies
between the requirements
• More empirical studies
- especially user studies
- relations between other NL artefacts, such as test cases
30

31. 31
Additional
Artefacts
Guidelines
https://sites.google.com/site/svvnarcia/