The Meetup Group "Thessaloniki Software Testing and QA" and the Meetup Group "Thessaloniki Verification and Testing Meetup" have organized a joint Meetup Meeting on December 12th in City Collage (Leontos Sofou 3 - Thessaloníki) at 6:00 pm. This is the Automated Test Generation presentation.

1. Automated Generation of Test
Cases for Software Verification:
Quality Assurance for Service-based
Applications
Dimitris Dranidis
International Faculty of the University of Sheffield
CITY College, Thessaloniki, Greece
December 2017
1

2. 2
Dr Dimitris Dranidis
 Senior Lecturer in Computer Science Department
 Academic director of postgraduate programmes:
 MSc in Advanced Software Engineering
 MSc in Management of Business, Innovation and
Technology
 Coordinator of the "Software Engineering & Service-Oriented
Technologies" research group
 http://www.city.academic.gr/csd/dranidis/

3. Personal research interests
 Formal methods
 Stream X-machines
 Model-based testing
 Testing & Run-time
Monitoring
 Service-based applications
 Certification registry for
SBAs
 Educational Informatics
 StudentUML
 Peer assessment
 Development of JSXM
 www.jsxm.org
 Model-based testing tool
(SXM testing theory)
3

4. Outline
 A world of services
 Motivation for automated test generation
 Model Based Testing
 State-based MBT
 Stream X-machines
 Testing Method
 The JSXM testing tool
 Features
4

5. Service-based Applications
service
service
consumer
consumes
service
Organization
boundaries
service
service
5

6. Traditional Testing approaches
are not effective!
 An application developer can
Guarantee the correctness of internal modules
Unit testing
Integration testing
Fix internal errors
Have control over changes
 … but should rely on correctness of external modules:
Testing of external modules is not enough….
Due to uncontrolled change
6

7. Need for Model-based testing -
Fully Automated Test generation
 A certification registry should be able to perform the tests
that guarantee service correctness.
The tests should not be provided by the provider
Tests might be incomplete (trust?)
Tests might depend on the implementation
 INSTEAD:
The tests must be produced based on the service
specification
Automated generation of tests based on a model
Generation is triggered by the registry
7

8. Model-based testing
Model System
Under Test
Abstract
Test Cases
describes
generate
generate
Executable
Test Cases
execute
8

9. Application of
Model-Based Testing on
Service-based Applications
9

10. Service publication &
Service verification by automated testing
service
Certification Registry
Service
spec tests
Automated
generation
10

11. Service advertisement &
Service animation (Sandbox Validation)
service
Certification Registry
Service
spec tests
consumer
Execution
of the model
11

12. Service monitoring
(run-time verification)
service
Certification Registry
Service
spec tests
consumer
Execution
of the service
Execution
of the model
12

13. Model-Based Testing based on
SXM Testing Theory
13

14. Finite Automata
 A Finite Automaton consists of:
a set of states Q
a set of inputs Σ
an initial state q0
a set of final states T
a function F: Q x Σ -> Q called the transition (next-
state) function.
 Can be represented as state diagrams in which
transitions are labeled with inputs.
14

15. Problems with FA
 State explosion!
 Focus only on control flow
15

16. Stream X-Machines
 A Stream X-Machine (SXM) is like a FA
with the following differences:
The machine has memory
transitions are labeled with functions that
• operate on inputs and memory
• update the memory and
• produce outputs
16

17. SXM definition
 An SXM consists of:
a set of states Q
a set of inputs Σ
a set of outputs Γ
a set M called memory
an initial state q0
an initial memory m0
a set Φ of processing functions, where each φ in Φ:
• φ : Σ x M -> Γ x M
a function F: Q x Φ -> Q called the transition function.
17

18. Processing functions
 A processing function consists of:
A guard condition depending on the input and
the memory
An effect which
• changes the memory and
• produces an output
18

19. SXM Account
Open(b, open()) = (openOut, 0)
Deposit(b, deposit(a)) = (depositOut, b + a)
Withdraw(b, withdraw(a)) =(withdrawOut, b - a) if a<b
WithdrawAll(b, withdraw(a)) =(withdrawOut, 0) if a=b
Close(b, close()) = (closeOut, b)
19
Initial Memory
b = 0

20. 20
SXM Testing
method

21. SXM Testing method:
Verification by testing
 A finite test set is produced
 based on a generalization of the W-method
 Execution of tests guarantees:
 Implementation conforms to the Model
21

22. Design-for-test conditions
 Taken from control theory
 Controllability
The ability to move a system in its state space
by admissible inputs
 Observability
The ability to determine the system state (or
behaviour) by using the inputs and the outputs
22

23. W-method (Chow)
 Applied on the associated finite automaton
A = (Φ, Q, F, q0)
 State cover S:
A set of sequences such that all states are reachable
from the initial state.
 Characterization set W:
a set of sequences for which any two distinct states of
the automaton are distinguishable.
23

24. W-method Test Set
 Test Set Yk of the associated automaton.
Yk = S.
Φk+1.
W
 A test set consists of sequences of processing functions
that are able to:
Reach every state by applying the state cover
sequences;
apply all tuples of functions of maximum length k+1;
and distinguish the resulting state by applying the
characterisation set sequences.
24

25. Discovered errors
 The SXM testing method can discover:
Missing states
Extra states
Missing transitions
Extra transitions
25

26. JSXM Model-based testing tool
http://www.jsxm.org
26

27. JSXM www.jsxm.org
 Model-based testing tool in Java
 Model as a Stream X-machine (SXM)
 Automated generation of test cases
 JSXM supports:
Model Animation (Model Validation)
Test Generation (Test Generation)
Test Transformation (Test execution)
27

28. Adapter
Adapter
JSXM overview
28
SXM Model
Java
application
Abstract
Test Cases
XML
generate transform
Junit
Test cases
PHPUnit
Test casestransform PHP
application
execute
execute
Animate

29. SXM Specification in XML:
Specification top structure
29
<SXM name="Account">
<states>
<initialState>
<transitions>
<memory>
<inputs>
<outputs>
<functions>
</SXM>

30. SXM Specification in XML:
States
30
<states>
<state name="initial" />
<state name="opened" />
<state name="closed" />
<state name="normal" />
</states>
<initialState state="initial" />

31. SXM Specification in XML:
Transitions labelled with functions
31
<transitions>
<transition from="initial" function="openF" to="opened" />
<transition from="opened" function="closeF" to="closed" />
<transition from="opened" function="depositF" to="normal" />
<transition from="normal" function="depositF" to="normal" />
<transition from="normal" function="withdrawN" to="normal" />
<transition from="normal" function="withdraw0" to="opened" />
</transitions>

32. SXM Specification in XML:
Inputs
32
<inputs>
<input name="open" />
<input name="close" />
<input name="deposit">
<arg name="amount" type=“xs:int" />
</input>
<input name="withdraw">
<arg name="amount" type=“xs:int" />
</input>
</inputs>

33. SXM Specification in XML:
Outputs
33
<outputs>
<output name="openOut" />
<output name="closeOut" />
<output name="depositOut">
<result name="amount" type=“xs:int" />
</output>
<output name="withdrawOut">
<result name="amount" type=“xs:int" />
</output>
</outputs>

34. SXM Specification in XML:
Memory (Java inline code)
34
<memory>
<declaration>
int balance;
</declaration>
<initial>
balance = 0;
</initial>
</memory>

35. SXM Specification in XML:
Functions (Java inline code)
35
<functions>
…
<function name="depositF" input="deposit“ ouput=“depositOut”>
<precondition>
deposit.get_amount > 0;
</precondition>
<effect>
balance = balance + deposit.get_amount;
depositOut.amount = deposit. get_amount;
</effect>
</function>
…
</functions>

36. JSXM Latest Features
 Constraint Logic Programming (CLP)
automates test values generation
 Complete GUI editor in Eclipse platform
Live editing of Diagram/XML specification
Validation of syntax
Quick fixes for frequent problems
36

37. JSXM Latest Features
 Automated generation of realizable State-Cover
Based on CLP integration
 Automated completion of the specification by
adding all necessary error self-transitions
Based on CLP integration
 Generation of Adapter classes for testing
37

38. Future research work
 Testing of Web applications
 Evaluation of JSXM in a large scale project
38

39. 39
Thank you!
Questions?