1.
STAMP
Software testing amplification to increase quality
in DevOps
1

2.
2
DevOps

3.
3
DevOps
Test assets are available
BUT
They are manually developed
and their quality is not sufficient

4.
4
DevOps
The main objective of STAMP is to
automatically transform existing
test assets in order to detect
regression bugs before
production and drive down the
cost of software testing.

5.
5
STAMP in DevOps

6.
Continuous test amplification
6

7.
Continuous test amplification
7
behav. diff.

8.
Continuous test amplification
8
behav. diff.

9.
Continuous test amplification
9
behav. diff. perf. diff.

10.
Continuous test amplification
10
behav. diff. perf. diff.

11.
Continuous test amplification
11
behav. diff. perf. diff.

12.
Unit test cases amplification
12
Program
P
Input space Output space
state

13.
Unit test cases amplification
13
Program
P
Input space Output space
state

14.
DSpot: amplification of JUnit test suites
14
original test suite and
program under test
update in the program
under test
P
DSpot
TS
P'
TS++

15.
DSpot
•First experiments with amplification
•Automatic transformation of Junit test cases
15
P
TS
P'
TS+
search-based
input space
amplification
observation
space
amplification
run and look
for differences
TS++

16.
Search-based input space amplification
• Mutate existing test cases
• transform literal values
• replace literal values by others found in the scope
• add new invocations to methods not well covered
• create new objects
• Search-based
• ‘stack’ transformations iteratively
• select the ones that have a good fitness (e.g. cover new
branches)
16

17.
Observation space amplification
• Remove all assertions from original test cases
• keep invocations that can be inside the assertion
• Add new observation points
• run all test cases on original program
• record all values that can be observed (getters, is_*, etc.)
• Filter new observation points
• several runs of all test cases with observation points on
the original program
• discard all points that vary (e.g., OS-dependent values,
IO, etc.)
17

18.
Example
18
public void testSearch() {
final ArrayStack<E> stack = makeObject();
stack.push(((E)("First Item")));
stack.push(((E)("Second Item")));
assertEquals("Top item is 'Second Item'", 1, stack.search("Second Item"));
assertEquals("Next Item is 'First Item'", 2, stack.search("First Item"));
assertEquals("Cannot find 'Missing Item'", -1, stack.search("Missing Item"));}
public void testSearch_cf5() {
final ArrayStack<E> stack = makeObject();
stack.push(((E)("First Item")));
stack.push(((E)("Second Item")));
E o_5 = stack.peek();
assertFalse(((java.lang.String)o_5).isEmpty());
E o_6 = stack.pop();
assertFalse(((java.lang.String)o_6).isEmpty());
Object o_7_0 = stack.search("Second Item");
Object o_9_0 = stack.search("First Item");
assertEquals("Cannot find 'Missing Item'", -1, stack.search("Missing Item"));
assertEquals(((java.lang.String)o_5).length(), 11);
assertEquals(((java.lang.String)o_6).length(), 11);}
new calls and asserts
extracted from asserts
new asserts

19.
Methodology
19
Unit test ampli
Config. test ampli
Online test ampli

20.
Methodology
20
Micro service
architecture
Unit test ampli
Config. test ampli
Online test ampli

21.
Methodology
21
Micro service
architecture
Unit test ampli
Config. test ampli
Online test ampli
Information
management
Software
quality
Cloud
computing
E-Health
Smart cities

22.
STAMP
•3-year project
•9 EU partners
•Full OSS
•Starts 01/12/16
22