How to verify software design and implementation against STPA safety requirements

1. Sommersemester 2017
Dr. Asim
Abdulkhaleq
STPA and Software
Verification

2. You can
copy, share and change,
film and photograph,
blog, live-blog and tweet
this presentation given that you attribute
it to its author and respect the rights and
licenses of its parts.
based on slides by @SMEasterbrook und @ethanwhite

3. STPA Approach

4. STPA Approach (Recall)

5. STPA Artefacts (Recall)
Diagrams
•Control Structure Diagram
•Process Model
The main
components used
in STPA
Data Lists
• Accidents
• Hazards
• System Goals
• Design and safety
Requirements
Data Tables
• Unsafe Control Actions
• Corresponding Safety
Constraints
• Causal Factors

6. STPA Data Flow (Recall)

7. Software Verification

8. Software Verification and Validation (V&V)
Verification
“Have we developed the software right?”
Validation
“Have we developed the right software?”
Boehm, Barry (1989). "Software Risk Management".

9. Software Verification
Dynamic Verification (Test, experimentation)
• Module Test
• Integration Test
• System Test
• Acceptance Test
• …
Static Verification (Analysis)
• Code Conventions Verification
• Bad Practices (anti-Pattern) Detection
• Software Metrics Calculation
• Formal Verification
• …
Approaches to Verification
Ghezzi, et al. Fundamentals of Software Engineering, Prentice Hall, ISBN 0-13-099183-X

10. Formal Verification
What we need to formally verify a software?
Software
Requirements
Software
Control Actions? Feedback?
Autonomous vehicle
modelled in graphical
notations e.g. State-
machine, stateflow.
Software Design
Models
written in natural
language e.g. English
Software Code
written in high-level
programming language
e.g. C, C++ or Java
Formalisation Transformation

11. Formal Specification and Verification
Specify and verify properties about how a system changes over time
Temporal Logics (TL)
“is a convenient formalism technique for specifying and verifying properties
of reactive systems”
Linear Temporal Logic (LTL)
“is a mathematical language for description linear-time properties”
Properties of interest include :
- Safety : “Nothing bad will happen”
- Liveness: “Something good will happen”.
- Fairness : “Independent subsystems make progress

12. Formal Specification and Verification
An LTL consists of:
- atomic proposition symbols [p, r, q,..], Boolean operators , and temporal
operators:
TL Operator LTL Operator Meaning Diagram
G ∅ means ”globally/always”. ∅ is true now
and forever in the future
X ∅ means “at the next time”. ∅ is true in
the next state
F ∅ means “eventually”. ∅ is true in some
future state
ℛ ψ R ∅
means “releases”. ∅ is true until and
including the point where ψ become
true. If ψ never becomes true, then ∅
must remain true forever
ψ U ∅
means ”until”. ψ is at least true until ∅
happens at the current or a future
position

13. Exercise: Formal Specification
Which LTL formula specifies the following requirement
“The train door software must not close (reopen) the
train door when there is a person in the doorway”?
1. LTL= G ((Person_In_Door_Way) -> G (ControlAction=Close))
2. LTL = G (!(Person_In_Door_Way) -> (ControlAction=Close) )
3. LTL = G ((Person_In_Door_Way) -> X ! (ControlAction=Close))
4. LTL = G ((Person_In_Door_Way) -> X! (ControlAction=Open))
3 Minutes

14. Software Formal Verification Process
How to verify my software design model/code?
Software
Requirements
Software
Design/Code
Software
Verification
Not Satisfied Satisfied
Errors
Track
Input
OutputNo
Results
check
YesNo

15. Software Model Checking
A technique to automatically verify whether the software meets its requirements.
At the design level
• The verification model can be constructed/extracted from the software
behaviour models such as finite state machine.
• Model checkers can be used at this level are:
1. SMV (Symbolic model Verifier) model checker
2. NuSMV Model checker
3. …
At the implementation level
• The verification model can be constructed/extracted directly from
software code written in C or Java.
• Software model checkers can be used at this level are:
1. SPIN (Simple Promela Interpreter)
2. …

16. SMV Language
Structure of the SMV model

17. SMV Language Example
Train door controller
MODULE main
VAR
states: {OpenDoor , CloseDoor};
Person_In_Door_Way: boolean;
ControlAction:{Open, Close};
ASSIGN
init (states):=OpenDoor;
init(Person_In_Door_Way):=bool(0);
next (states):=case
states=OpenDoor & !(Person_In_Door_Way) : CloseDoor;
states=CloseDoor& (Person_In_Door_Way): OpenDoor;
TRUE: {CloseDoor ,OpenDoor};
esac;
next(ControlAction):=case
states=OpenDoor: Open;
states=OpenDoor & !(Person_In_Door_Way) : Close;
states=CloseDoor& (Person_In_Door_Way): Open;
TRUE:{Open, Close};
esac;
Software behavioural model (stateflow) SMV Model (verification model)

18. NuSMV Model Checker
•It is an open source tool developed by ITC-IRST and
UniTN 1999
•It is an extension and re-implementation of SMV
model checker.
•It based on Binary Decision Diagram (BDD) to verify
finite state machine against its specifications
expressed in LTL

19. Exercise: Formal Specification
Which LTL formula is satisfied by the SMV model of
the train door controller ?
1. LTL= G ((Person_In_Door_Way) -> G (ControlAction=Close))
2. LTL = G (!(Person_In_Door_Way) -> (ControlAction=Close) )
3. LTL = G ((Person_In_Door_Way) -> X ! (ControlAction=Close))
4. LTL = G ((Person_In_Door_Way) -> X ! (ControlAction=Open))
3 Minutes

20. NuSMV Model Checker Result
“it contains information that shows why the given LTL is not satisfied”
Counterexample
-- specification G (Person_In_Door_Way -> G
ControlAction = Close) is false
-- as demonstrated by the following execution sequence
Trace Description: LTL Counterexample
Trace Type: Counterexample
-> State: 1.1 <-
states = OpenDoor
Person_In_Door_Way = FALSE
ControlAction = Open
-> State: 1.2 <-
states = CloseDoor
Person_In_Door_Way = TRUE
-- Loop starts here
-> State: 1.3 <-
states = OpenDoor
Person_In_Door_Way = FALSE
-> State: 1.4 <-
states = CloseDoor
-> State: 1.5 <-
states = OpenDoor
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.

21. Formal Verification Challenges
•Formal verification focuses on proving the functional
correctness of software.
•It can not make the software safe or reduce the risk.
•It needs expert users
•The requirements can not be directly verified by
software model checker
•State explosion problem

22. STPA in Software
Verification

23. STPA in Software Verification
How to verify software against STPA results?
Software
Control Actions? Feedback?
Train Door
satisfied
not satisfied
STPA Safety
Requirements
Safety constraints
STPA Results
Software Verification
(Formal Verification & Testing)

24. STPA SwISs: A System-Theoretic Safety Engineering Approach for
Software-Intensive Systems

25. Detailed STPA SwISs Approach

26. Exercise: Apply STPA to Train Door System
Disuse with your colleague how to apply STPA to the
train door system and do the following:
1. Write one accident that the software controller of train door
can contribute in.
2. Write 3 hazards which maybe cause this accident.
3. Translate the identified hazards into the safety constraints.
4. Draw the control structure diagram of the train door system.
5. Identify one unsafe control action and translate it into a
corresponding safety constraint.
10 Minutes

27. Deriving the STPA Software Safety Requirements
Accidents
• AC1- A person is injured
while the train closed the
door.
• ….
Hazards
• H1- Door close on a person
in the doorway.
• …
System-level safety constraint
• The train door controller
must not close the door
while a person is in the
doorway
• …
safety control structure diagram of train door system

28. Unsafe Control Actions
Control Action
• C1- Close
• ….
Unsafe Control Actions
• UCA1.1- Train door controller
closes the door while a person in
the doorway
• …
Corresponding Safety Constraints
• CSC 1.1- Train door controller
must not close the door while a
person or object is in the
doorway
• …

29. Process Model & Variables

30. Generating the Unsafe Scenarios & Requirements
Unsafe Control action
• UCA1.1- Train door controller closes the door while a person in the doorway
Basic Scenario
• RUCA1.1- The door software controller provided the control action close door while the train is
stopped, train position is aligned, door state is open and a person is in the doorway.
Refined software safety constraint
• RSC1.1- The door software controller must not provide the control action close door while the train
is stopped, train position is aligned, door state is open and a person is in the doorway.

31. Causal Factors & Causal Scenarios
How each unsafe control action could occur in the system?

32. Formalisation of STPA
Results

33. Formalisation of STPA Results
Providing or not providing a control action (CA) is based on the occurrence of
the set of values of process model variables and higher inputs (CS).
• Rule 1: When CS occur in the execution path, the software must not (!) provide CA.
Then, LTL formula can be expressed as: LTL = G ( CS → ! CA).
• Rule 2: When CS occur in the execution path, the software must provide CA at the
next step. then LTL formula can be expressed as:LTL = G ( CS → X CA)
• Rule 3: The software must always not (!) provide CA too early (U) before CS the
occurrence of CS still not become true. Then, the LTL formula can be expressed as:
LTL = G ((CA →CS) & (!CA U CS))
• Rule 4: The software must always not (!) provide CA too late (U) while the occurrences
of the critical set of combinations CS has become previously true in the execution
path. Then, the LTL formula can be expressed as: LTL = G ((CS → CA) & (!CS U
CA))

34. Exercise: Formalisation of STPA Results
Write the corresponding LTL formula for the following
software safety requirements:
1. “The train door software controller must not
provide the control action close door while the
train is stopped, train position is aligned, door
state is open and a person is in the doorway”
2. The train door software controller must provide
the control action open door when there is a
person in doorway.
5 Minutes

35. Verify STPA Results
within XSTAMPP

36. XSTAMPP
•It is an open source tool developed at our institute
through student project and job (2013-2017).
•Designed specially to serve the widespread adoption
and use of STAMP in different areas.
•It extended with an plugin called STPA verifier to
support the software safety and verification (formal
verification and testing activities).
www.xstampp.de

37. STPA Verifier Process
www.xstampp.de

38. STPA Verifier Main Window

39. Generating Safety-
based Test Case from
STPA Results

40. Detailed STPA SwISs Approach (Recall)

41. Safety-based Test Case Generation Algorithm
Safe Behavioral
Model
Verify
?
Safety-based Test
Cases
traverse
not satisfied
satisfied
modify
export
LTL formulae STPA
Results
Traceability
matrix
transform
check
SMV Model
Safe Test Model
model
Test case sheet
transform
Generating Safety-based Test Cases
1 Modelling STPA Results
2 Transforming into a Formal Model
3 Checking Correctness with Model Checker
4 Generating Runnable Safe Test Model
5
Algorithm

42. Modelling STPA Results
A Safe Software behavioural Model

43. Mapping STPA Results into Stateflow Model

44. Safe behavioural model of train door controller
Control structure diagram
with process model
Safe behavioural model of
train door controller

45. STPA Test Cases Generator

46. References
ISBN-10: 3736994923
ISBN-13: 978-3736994928
ISBN: 9780262026499

47. Dr. Asim Abdulkhaleq
e-mail asim.abdulkhaleq@informatik.uni-
stuttgart.de
phone +49 (0) 711 685-88458
WWW www.iste.uni-stuttgart.de/se
Twitter @AbdulkhaleqAsim
Institute of Software Technology

48. Pictures Used in this Slide Deck
• V-Model https://commons.wikimedia.org/wiki/File:V-model-en.png
• NuSMV logo: http://nusmv.fbk.eu
• Train door photo is taken from http://www.bbc.com/news/uk-england-
tyne-24634884
• Principle of model checker photo : https://mitpress.mit.edu/books/principles-
model-checking