This document discusses security requirements for web-based, multi-device systems. It presents an approach to capturing security requirements using misuse case specifications in natural language. The approach involves defining misuse cases that describe potential attacks and security use cases that specify security countermeasures. The document then discusses how these security requirements specified in natural language can be used to automatically generate executable security test cases. Specifically, it describes how natural language processing techniques can be applied to identify test inputs, operations, and oracles from the misuse case specifications in order to generate code for vulnerability testing.

1. .lusoftware verification & validation
VVS
Testing Security and Privacy
Requirements
Lionel Briand
NASAC 2018, China

2. Context: Web-based, Multi-device Systems
2
• Include multiple and different devices that manage personal information
• Typical for many services (e.g., personal training, home-banking, music-streaming, etc. )
• Case study: EDLAH2 project, active and assisted living for the elderly
• Multiple attack surfaces, e.g., parameters, URLs, files, programs

3. 42%
32%
9%
4%
3%
3%
3%
2%
2%
Code Injection
Manipulated data…
Collect and analyze…
Indicator
Employ…
Manipulate system…
Subvert access…
Abuse existing…
Engage in…
X-Force Threat Intelligence Index 2017
3
https://www.ibm.com/security/xforce/
More than 40% of all
attacks were injection
attacks (e.g., SQLi)

4. Challenges
4
• Capture security and privacy requirements
•Both properties to preserve (e.g., confidentiality of personal data)
and possible attacks to be prevented
•Structured and precise, e.g., to enable compliance analysis and
test automation
•Coherent with functional requirements elicitation practice
(i.e., use case-driven in EDLAH2)
• Ensure, in a verifiable manner, that security and privacy
requirements have been properly implemented
• Usually based on manual testing, which is error prone and time
consuming

5. Research Objectives
5
Define a method for capturing security
requirements in a form that is both practical
and amenable to test automation
Define an automated testing methodology
to ensure the compliance of the software
with its security requirements

6. Security Requirements:
A Use Case-driven Approach
6

7. Objectives
7
• Security requirements should be “usable” by all
stakeholders
•Technical and, to some extent, non-technical roles
•Technical roles: IT people who cannot handle formal
representations
• Security requirements with integrated with functional
requirements (Use Cases)
• Security requirements should be automatically analyzable
for multiple purposes, e.g., automated security testing

8. Misuse
Case
Specifications
Use Case
Specifications
Security
Use Case
Specifications
Misuse
Case
Diagram
Mitigation
Schemes
Restricted Misuse Case Modeling
RMCM
Based on templates with keywords
8

9. Patient
Access
Discussion Boards
Provide
Health Data
Get Fitter
by Walking
Monitor
Patient Progress
Configure
Patient Tablet
Login
View
Patient Information
Malicious
User
Malicious
App
Bypass
Authorization
Schema
Collect
Credentials
Extract Data
From Insecure
Data Storage
Get Access
with SQLi
«threaten»
«threaten»
«threaten»
Validate
Data Access
Rights
«include»
«mitigate»
Family
Member
EDLAH2

10. RMCM Misuse Case Template
10
•Based on RUCM [Yue’13], a template for capturing use
case specifications
•Restrictions rules to avoid ambiguity
•Keywords to enable precise natural language processing (NLP)
•Compliance checked with NLP
•Not originally designed for security and privacy
•Extension: Restricted Misuse Case Modeling (RMCM)

11. A Generic Example of Misuse Case
Specification Header
MISUSE CASE: Bypass Authorization Schema
Description: The MALICIOUS user accesses resources that
are dedicated to a user with a different role.
Precondition: The MALICIOUS user has access to one or
more accounts on the system and a list of resources (URLs)
that she should not be able to access with these accounts.
Primary Actor: MALICIOUS user
Threats: Monitor Patient Progress, View Patient Information,
Configure Patient Tablet
Assets: Client DATA
11

12. A Generic Example Misuse Case
Specification Header
MISUSE CASE: Bypass Authorization Schema
Description: The MALICIOUS user accesses resources that
are dedicated to a user with a different role.
Precondition: The MALICIOUS user has access to one or
more accounts on the system and a list of resources (URLs)
that she should not be able to access with these accounts.
Primary Actor: MALICIOUS user
Threats: Monitor Patient Progress, View Patient Information,
Configure Patient Tablet
Assets: Client DATA
Indicates malicious actor
Indicates security-sensitive data
12

13. Example Misuse Case Specification Flows
Basic Threat Flow:
1. FOREACH role
2. The MALICIOUS user SENDS username and password TO the system
through the login page
3. FOREACH resource
4. The MALICIOUS user REQUESTS the resource FROM the system.
5. The system SENDS a response page TO the MALICIOUS user.
6. The MALICIOUS user EXPLOITS the system using the response page.
7. ENDFOR
8. ENDFOR
Postcondition: The MALICIOUS user has accessed a resource dedicated
to another user with different role
Specific Alternative Threat Flow (SATF1)
RFS 4
1. IF the URL includes a role parameter THEN
2. The MALICIOUS user updates the role value with his role
3. RESUME STEP 5.
…
Nominal scenario for a malicious actor to
successfully harm the system

14. Example Misuse Case Specification Flows
Basic Threat Flow:
1. FOREACH role
2. The MALICIOUS user SENDS username and password TO the system
through the login page
3. FOREACH resource
4. The MALICIOUS user REQUESTS the resource FROM the system.
5. The system SENDS a response page TO the MALICIOUS user.
6. The MALICIOUS user EXPLOITS the system using the response page.
7. ENDFOR
8. ENDFOR
Postcondition: The MALICIOUS user has accessed a resource dedicated
to another user with different role
Specific Alternative Threat Flow (SATF1)
RFS 4
1. IF the URL includes a role parameter THEN
2. The MALICIOUS user updates the role value with his role
3. RESUME STEP 5.
…
Control flow
Input and output steps
Exploit information exposed in
error or exception messages
Unwanted impact on asset

15. Basic Threat Flow:
1. FOREACH role
2. The MALICIOUS user SENDS username and password TO the system
through the login page
3. FOREACH resource
4. The MALICIOUS user REQUESTS the resource FROM the system.
5. The system SENDS a response page TO the MALICIOUS user.
6. The MALICIOUS user EXPLOITS the system using the response page.
7. ENDFOR
8. ENDFOR
Postcondition: The MALICIOUS user has accessed a resource dedicated
to another user with different role
Specific Alternative Threat Flow (SATF1)
RFS 4
1. IF the URL includes a role parameter THEN
2. The MALICIOUS user updates the role value with his role
3. RESUME STEP 5.
...
Alternative attack scenario:
includes the keyword ‘Threat’
Example Misuse Case Specification Flows
Specific alternative threat flow branching from
step 4 in basic threat flow

16. Basic Threat Flow:
1. FOREACH role
2. The MALICIOUS user SENDS username and password TO the system
through the login page
3. FOREACH resource
4. The MALICIOUS user REQUESTS the resource FROM the system.
5. The system SENDS a response page TO the MALICIOUS user.
6. The MALICIOUS user EXPLOITS the system using the response page.
7. ENDFOR
8. ENDFOR
Postcondition: The MALICIOUS user has accessed a resource dedicated
to another user with different role
Specific Alternative Flow (SAF1)
RFS 5
1. IF the response page contains an error message THEN
2. RESUME STEP 7.
3. ENDIF.
Postcondition: The MALICIOUS user cannot access the resource
Failed attack scenario:
‘Alternative Flow’ used instead of
‘Alternative Threat Flow’.
Alternative flows always begin with a conditional
statement: simplify the identification of the
conditions that trigger the alternative behavior.
Example Misuse Case Specification Flows

17. Security Use Case Specification
17
SECURITY USE CASE: Validate Data Access Rights
Description: The system verifies if the access to the data of a certain patient
can be granted to a given user.
Precondition: A user has requested patient DATA
Compliance: ISO/IEC 27001:2013 clause A.9.4
Mitigate: Bypass Authorization Schema
Basic Flow
1. The system VALIDATES THAT the user is a carer.
2. The system VALIDATES THAT the user belongs to the
group of carers of the patient.
3. The system SENDS the requested data TO the user.
Postcondition: The DATA access is granted.
Specific Alternative Flow (SAF2):
RFS 1
1. ABORT.
Postcondition: Patient DATA access is denied.
Counter-measure for misuse case(s)
Steps performed to
mitigate attacks in
misuse cases
Traceability to provisions
in standards

18. Security Use Case Specification
18
SECURITY USE CASE: Validate Data Access Rights
Description: The system verifies if the access to the data of a certain patient
can be granted to a given user.
Precondition: A user has requested patient DATA
Compliance: ISO/IEC 27001:2013 clause A.9.4
Mitigate: Bypass Authorization Schema
Basic Flow
1. The system VALIDATES THAT the user is a carer.
2. The system VALIDATES THAT the user belongs to the
group of carers of the patient.
3. The system SENDS the requested data TO the user.
Postcondition: The DATA access is granted.
Specific Alternative Flow (SAF2):
RFS 1
1. ABORT.
Postcondition: Patient DATA access is denied
User not a carer
Check condition
Condition violated

19. Misuse
Case
Specifications
Use Case
Specifications
Security
Use Case
Specifications
Misuse
Case
Diagram
Mitigation
Schemes
Toolset
Checks conformance of specifications with RMCM syntax
Checks consistency of diagrams and specifications
• Use cases should appear both in diagram and specifications
• Dependencies should appear both in diagram and specifications
• https://sites.google.com/site/rmcmverifier/
+ Conformance & Consistency Checker
based on NLP
+ UML profiles

20. Requirements-Driven
Security Testing
21

21. Misuse
Case
Specifications
Security
Use Case
Specifications
Security
Functional
Testing
Security
Vulnerability
Testing
Automated Generation of
Executable Test Cases
Validating whether the
specified security
properties are
implemented correctly
Addressing the
identification of
system
vulnerabilities
Automatic generation of executable test
cases from security requirements
Benefits of automated generation:
• Automated generation reduces
development costs
• Ensures coverage and traceability
Focus on security vulnerability
testing in this presentation
[Wang‘15, Wang‘18]

22. Objective
23
Misuse
Case
Specifications
Executable
Security
Vulnerability
Test Cases
Automated Generation
based on Natural
Language Processing
i.e., natural language
description of attacks

23. Example Misuse Case Specification
(2 flows)
24
Basic Threat Flow:
1. FOREACH role
2. The MALICIOUS user sends username and password to the system
through the login page
3. FOREACH resource
4. The MALICIOUS user requests the resource from the system
5. The system sends a response page to the MALICIOUS user
6. The MALICIOUS user EXPLOITS the system using the response page
7. ENDFOR
8. ENDFOR
Postcondition: The MALICIOUS user has executed a function
dedicated to another user with different role
Specific Alternative Flow (SAF1)
RFS 5
1. IF the response page contains an error message THEN
2. RESUME STEP 7
3. ENDIF
Postcondition: The MALICIOUS user cannot access the resource

24. Natural Language and Test Generation
25
• Identify input entities (e.g., ‘role’, ‘password’), input relationships (e.g., each
‘username’ is associated to a ‘role’), and values to be assigned to input
entities.
• Identify the instructions (e.g., API calls) that perform the operations
indicated in the use case specifications steps.
• Generate oracles/verdicts from conditional sentences describing when the
attack is successful.
Basic Threat Flow:
1. FOREACH role
2. The MALICIOUS user SENDS username and password TO the system
through the login page
3. FOREACH resource
4. The MALICIOUS user REQUESTS the resource FROM the system.
..
1. IF the response page contains an error message THEN
input entities
operation
conditional
sentence

25. Working Assumptions
•RMCM is applied
•A test driver API is available
•API operation and parameters are “textually similar” to phrases in use
case specifications
•Natural language processing techniques (i.e., semantic role labeling)
are accurate at identifying methods to call, inputs …
parameters["passwd"] = role["password"]
parameters["user"] = role["username"]
system.send("login page", parameters)
“The MALICIOUS user sends username and password to the system”
caller of a method
method to
call
parameters
instance to
invoke
System::send(String page, Dictionary parameters);

26. Expected Generated Code
27
roleIter = input["role"].__iter__()
while roleIter.__hasNext__():
role = roleIter.__next__()
arguments["password"] = role["password"]
arguments["username"] = role["username"]
system.send("login page", arguments)
resourceIter = role["resource"].__iter__()
while resourceIter.__hasNext__():
resource = resourceIter.__next__()
system.request(resource)
responsePage = system.responsePage
if not responsePage.contain(resource["error_message"]):
# the test case fails: the attacker exploits the vulnerability
maliciousUser.exploit()
else:
# the test case passes and the attacker lose
maliciousUser.abort(“malicious user cannot access resource”)
For every role
configured on the system
Login as a user
with that role
Request a resource
(not available to a user with that role)
If the response does not contain
an error message
we have accessed the resource
and can exploit the system
Otherwise the attacker
cannot access the resource

27. Expected Generated Code
28
roleIter = input["role"].__iter__()
while roleIter.__hasNext__():
role = roleIter.__next__()
arguments["password"] = role["password"]
arguments["username"] = role["username"]
system.send("login page", arguments)
resourceIter = role["resource"].__iter__()
while resourceIter.__hasNext__():
resource = resourceIter.__next__()
system.request(resource)
responsePage = system.responsePage
if not responsePage.contain(resource["error_message"]):
# the test case fails: the attacker exploits the vulnerability
maliciousUser.exploit()
else:
# the test case passes and the attacker lose
maliciousUser.abort(“malicious user cannot access resource”)
For every role
configured on the system
Login as a user
with that role
Request a resource
(not available to a user with that role)
If the response does not contain
an error message
we have accessed the resource
and can exploit the system
Otherwise the attacker
cannot access the resource
Generated code includes
input processing,
variable declarations,
cycles, conditions,
method calls, assignments,
oracles

28. MCP
29
Tool: https://sntsvv.github.io/MCP/
Test Driver API
provided by MCP
(possibly extended
by engineers)
Phase 1: Map the test driver API to the MCP ontology
Misuse Case Specifications
In Natural Language
Bypass Authorization
Step 1:…
Step 2:…
(A) Initial ontology provided by
MCP (models programming
language concepts).
Class
AttributeMethod
(B) Ontology including information
about the test driver API.
«Class»
HttpTest
«Class»
System
«Method»
send
Class
Method
Attribute
Phase 2: Generate Misuse Case Models
Step 1
Step 3Step 5
(C) Misuse Case Model
capturing control flow.
Phase 3: Identify Test Inputs
(D) Ontology
updated with
individuals capturing
the relations
between inputs.
«Key»
role
«Key»
username
«Key»
password
«Dictionary»
inputs
(E) Test Input files
Inputs.json
Phase 4: Generate Executable Test Cases
(G) Executable Test Cases.
reuseInvitation.py
guessUserAccount.py
bypassAuthorization.py
(F) Ontology updated with information about
instance variables in the scope of test case lines.
«Class»
HttpTest
«Class»
System
«Variable»
system
Class
Method
Attribute
«Class»
BPA
«Variable»
this
«Scope»

29. Step 3.1: Identify input entities
Basic Threat Flow:
1. FOREACH role
2. The MALICIOUS user SENDS
username and password TO
the system through the login page
3. FOREACH resource
4. The MALICIOUS user REQUESTS
the resource FROM the system.
..
6. IF the response page contains an
error message THEN
30
• Determine which sentences
describe activities where inputs
are provided to the system
• Determine which phrases are
the inputs

30. Step 3.1: Identify input entities
Basic Threat Flow:
1. FOREACH role
2. The MALICIOUS user SENDS
username and password TO
the system through the login page
3. FOREACH resource
4. The MALICIOUS user REQUESTS
the resource FROM the system.
..
6. IF the response page contains an
error message THEN
• Rely on Semantic Role Labeling
• Inputs are the terms affected by
the verb, (i.e., the items
provided by an actor to the
system)
31

31. Semantic Role Labeling (SRL)
32
“The MALICIOUS user sends the username to the system”
actor affected
by the activity
actor performing
an activity
verb
SRL: Automatically determines the semantic roles of
phrases in sentences
destination

32. Semantic Role Labeling (SRL)
33
“The MALICIOUS user sends the username to the system”
actor affected
by the activity
actor performing
an activity
verb
SRL: Automatically determines the semantic roles of
phrases in sentences
destination
In an input sentence when the destination is the system:
The input is the actor affected by the activity.

33. Step 3.4: Input values are automatically generated
when an attack-specific keyword appears in the
use case specifications
MISUSE CASE: Get default credentials
Basic Threat Flow
1. DO
2. The MALICIOUS user SENDS AUTHENTICATION VALUES TO the system
through the login page in the username and password fields of the login page.
3. The system SENDS the response page TO the malicious user
4. …
Postcondition: The MALICIOUS user has been logged into the system by using a
default credential
…..
34
Keywords matches input generation strategy
• Inputs generated by expert or automatically (preferably)
• We have conducted research on test input generation strategies for standard
attacks, e.g., XML injections

34. Generating Test Inputs for
Effective Attacks
35

35. 36
Example: Generating XML Injection
Attacks for Front-end Web
Applications

36. Testing the Front-end (XMLi)
37
Input
Strings

37. Security Mechanisms in Front-end
Web Applications
• Input Sanitization: rejects inputs
containing malicious characters
(e.g., <)
• Input Validation: converts
malicious inputs to valid ones
(e.g., deleting XML tags)
• Other transformations: domain
specific transformation (e.g.,
JSON to XML, calculating age)
38
Input
Strings

38. Testing of the Front-end WAs
39
Does the front-end system (SUT) allow the
generation of XML injection attacks?
YES
The front-end
is vulnerable
NO
The front-end
is secure

39. Testing of the Front-end WAs
40
<user>
<username>Tom</username>
<password>m1U9q10</password>
<role>user</role>
<mail>role=Adm+ tom@uni.lu</mail>
</user>
Step 1: Create malicious XML messages
Step 2: Verify whether the SUT can generate them
Malicous XML message
Search for
Input String

40. Generating Malicious Messages
Grammar-based Generation: automatically generating malicious
messages from legitimate messages for different type of XML
injection attacks
41
Our tool SOLMI (ISSTA'16)
Example of XML message
generated by SOLMI

41. Searching for Input Strings
42
<user>
<username>Tom</username>
<password>m1U9q10</password>
<role>user</role>
<mail>role=Adm+ tom@uni.lu</mail>
</user>
Malicous XML message
Candidate
Input String
• The front-end web application (SUT) is a black-box
• The search space is huge: all possible input strings (I1, .., In)
• Genetic algorithms

42. Searching for Input Strings
43
Evaluation
Selection
Crossover
Mutation
Genetic
Algorithm
Initial
Solutions Random Strings
Email:“role=Adm”
+tom@uni.lu
Usr: Tom
Psw: m1U9q10

43. Searching for Input Strings
44
Evaluation
Selection
Crossover
Mutation
Genetic
Algorithm
Initial
Solutions Random Strings
Email:“role=Adm”
+tom@uni.lu
Usr: Tom
Psw: m1U9q10
Target Edit
Distance
XMLXML

44. Searching for Input Strings
45
Evaluation
Selection
Crossover
Mutation
Genetic
Algorithm
Initial
Solutions Changed Input Strings
Email:“role=…”
+tom@uni.lu
Usr: Tom
Psw: m1U9q10
XML
XML
XML
New XML
messages

45. Results
46
Application
% TO Coverage
(search)
% TO Coverage
(random)
Avg. Exec time per TO
(min-max) in mins
SBANK
(Insecure)
100 0 10-31
SSBANK
(Secure)
36.73 0 11-25
XMLMao
(open source)
100 0 5-7
M
(Industrial App)
25 0 32
Note: Each experiment was repeated 10 Times to account for randomness.

46. 47
Detecting Successful Attacks?

47. 48
Detecting Malicious SQL Statements

48. Objectives & Strategy
49
• Oracle for vulnerability testing (e.g., are SQL injection
attacks successful?)
• Can also be used as database firewall
• Use of machine learning (unsupervised)
• Clustering: Model of legitimate SQL statements
• Detection of SQL injections: Are SQL statements clearly
distinct from the model of legitimate statements?

49. Detecting SQL Injection Attacks
50
Parsing Pruning
Edit
Distance
Clustering
SQL
Legitimate
Execution Logs
Phase 1: Training to model legitimate statements
SQL
Security
Testing Logs
Parsing Pruning
Phase 2: Detection of SQL injections

50. Detection Phase
51
Incoming
SQL Statement 1
Incoming
SQL Statement 2

51. Detection Phase
52
Incoming
SQL Statement 1
Incoming
SQL Statement 2
REJECT
APPROVE

52. Results
• Perfect recall: We catch all malicious SQL statements
• Low false positive rate: Very few legitimate statements get
flagged as malicious (~ 0.1%)
53

53. Summary
54

54. Requirements-Driven Testing
55
• Security and privacy requirements expressed as restricted
use case specifications plus keywords.
• Trade-off between free-form, natural language requirements
and analyzable requirements.
• Requirements-driven security testing to generate complex
attack scenarios is automated.
• Explicit testing rationale and traceability useful in the
context of required and verifiable compliance with standards
and regulations

55. Generating Test Input Data
56
•Test input generation => standard attack generation
(e.g., SQLi)
•Re-express the attack generation problem as a search
problem, using evolutionary computing
•Common challenge: Effective way to assess how close
we are to a successful attack and to detect successful
attacks automatically

56. Artificial Intelligence is Key
57
A number of key AI technologies are recurring in many
automated solutions for security testing:
•Natural Language Processing, e.g., SRL
•Machine learning, e.g., cluster analysis
•Evolutionary computing, e.g., genetic algorithms

57. Acknowledgments
58
•Fabrizio Pastore
•Annibale Panichella
•Sadeeq Jan
•Dennis Appelt
•Phu Mai
•Mariano Ceccato
•…

58. References
59
•Mai et al., “Modeling Security and Privacy Requirements: a
Use Case-Driven Approach”, IST journal (Elsevier), 2018
•Mai et al., “A Natural Language Programming Approach for
Requirements-based Security Testing”, ISSRE 2018
•C. Wang et al., Automatic Generation of System Test Cases
from Use Case Specifications , ISSTA 2015
•C. Wang et al., Automated Generation of Constraints from Use
Case Specifications to Support System Testing, ICST 2018
•T. Yue et al., “Facilitating the transition from use case models
to analysis models: Approach and experiments”, ACM TOSEM
2013

59. References
60
•Jan et al., “Automatic Generation of Tests to Exploit XML Injection
Vulnerabilities in Web Applications”, IEEE Transaction on Software Engineering
(TSE), 2018
•Appelt et al., “A Machine Learning-Driven Evolutionary Approach for Testing
Web Application Firewalls, “IEEE Transaction on Reliability (TR), 2018
•Appelt et al., “Automatically Repairing Web Application Firewalls Based on
Successful SQL Injection Attacks”, IEEE 28th International Symposium on
Software Reliability Engineering (ISSRE 2017)
•Jan et al., “Search-based Testing Approach for XML Injection Vulnerabilities in
Web Applications”, Proc. of the 10th IEEE International Conference on Software
Testing, Verification and validation (ICST 2017)
•Jan et al., “Automated and Effective Testing of Web Services for XML Injection
Attacks”, Symposium on Software Testing and Analysis (ISSTA 2016)

60. References
61
•Ceccato et al., “SOFIA: An Automated Security Oracle for Black-Box Testing of SQL-
Injection Vulnerabilities”, 31th IEEE/ACM International Conference on Automated
Software Engineering (ASE 2016)
•Jan et al., “Known XML Vulnerabilities Are Still a Threat to Popular Parsers and Open
Source Systems”, IEEE International Conference on Software Quality, Reliability &
Security (QSR 2015)
•Appelt et al., “Behind an Application Firewall, Are We Safe from SQL Injection
Attacks?”, 8th International Conference on Software Testing, Verification, and
Validation (ICST 2015)
•Appelt et al., “Automated Testing for SQL Injection Vulnerabilities: An Input Mutation
Approach”, International Symposium on Software Testing and Analysis (ISSTA 2014)

61. .lusoftware verification & validation
VVS
Testing Security and Privacy
Requirements
Lionel Briand
NASAC 2018, China