The document discusses automatic test generation for space applications. It describes a master's thesis that aims to automatically generate tests for an operational simulator using model-based and constraint-based testing. The thesis will extract UML diagrams and invariants from existing code and relate them to requirements. Tools like Korat and Pex were studied, with Korat able to generate complex data structures that satisfy invariants, while Pex can achieve high code coverage but does not maintain data structure invariants. The work will combine static analysis and data structure generation to automatically test the operational simulator.

1. Automatic Test Generation for Space
Ulisses Costa Daniela da Cruz Pedro Rangel Henriques
SLATE’12 - Symposium on Languages, Applications and Technologies
June 21, 2012
Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space

2. Context
Problem
VST (Visionspace Technologies) provides services related to testing
for ESA and wants to automate the test generation for the
Operational Simulator platform.
This presentation appears in the context of a master’s thesis that
aims at:
Generate automatically tests for the Operational Simulator
Generate unit tests for the Operational Simulator language –
C++
Parametrize the size of the generated data structures and be
able to configure other attributes
Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space

3. Motivation
Extract UML and OCL from the existing code
Extract tests from the existing code
Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space

4. OCL Inference
OCL is a language used to describe logic properties about UML
models, typically in the form of invariants.
The first step is to extract interesting invariants from the code and
match them with requirements.
Generate UML diagrams from the existing code (easy)
Infer code invariants (hard)
Relate the discovered invariants with the UML diagrams
Relate the discovered diagrams with requirements
Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space

5. White vs. Black Box Testing
Types of tests regarding the knowledge about the code
White Box , there is knowledge about the code, and this
knowledge is used to perform the test generation.
Black Box , there is only knowledge about the requirements and
about how each component should behave.
Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space

6. Approaches 1/2
Specification-based Generation Testing , aka Model Based Testing
consists in testing a program based on the program
specification or on the program model. Test cases
can be generated from the specification, without
consider the code.
Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space

7. Approaches 2/2
Constraint-based Generation Testing , can be used to select test
cases that meet some variable restrictions. When
combined with symbolic execution, gathers
restrictions along the different paths in the CFG. It is
possible to solve these restrictions and generate test
cases.
Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space

8. Current state
By now, we studied different approaches and tools, the more
important to our goal are:
Korat, is a mature framework to automatically construct
complex structures for JAVA
Pex is a White-box testing framework from Microsoft tool
that tries to give total code coverage
Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space

9. Studied Tools - Pex
1 public class Program {
2 public static int BSearch ( int x , int n ) {
3 return BinarySearch (x , 0 , n ) ;
4 }
5 static int BinarySearch ( int x , int lo , int hi ) {
6 while ( lo < hi ) {
7 int mid = ( lo + hi ) /2;
8 Debug . Assert ( mid >= lo && mid < hi ) ;
9 if ( x < mid ) { hi = mid ; } else { lo = mid +1; }
10 }
11 return lo ;
12 }
13 }
Result x n result Output/Exception
0 0 0
0 1 1
0 3 1
1073741888 1719676992 TraceAssertionException
1 6 2
50 96 51
Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space

10. Studied Tools - Korat
1 public class LinkedList T {
2 public static class LinkedListElement T {
3 public T Data ;
4 public LinkedListElement T Prev ;
5 public LinkedListElement T Next ;
6 }
7 private LinkedListElement T Head ;
8 private LinkedListElement T Tail ;
9 private int size ;
10 }
LinkedList class invariants (circular doubly linked list):
∀ l : l ∈ LinkedList : Head(l) ≡ null ∨ Tail(l) ≡ null ⇔ size(l) ≡ 0 (1)
∀ l : l ∈ LinkedList : Tail(l).Next ≡ null (2)
∀ l : l ∈ LinkedList : Head(l).Prev ≡ null (3)
∀ l : l ∈ LinkedList : size(l) ≡ 1 ⇔ Head(l) ≡ Tail(l) (4)
∀ l : l ∈ LinkedList : ∀ e1 , e2 : {e1 , e2 } ⊆ l : ∃ e : e ∈ l : e1 .Next ≡ e ∧ e2 .Prev ≡ e
⊆ ∈ (5)
∀ l : l ∈ LinkedList : ∀ e1 , e2 : {e1 , e2 } ⊆ l : e1 ≡ e2 ⇒ i(e1 ) ≡ i(e2 )
⊆ (6)
Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space

11. Studied Tools - Pex - LinkedList
(a) LinkedList instance (b) LinkedList instance
generated by Pex to test generated by Pex to test
the method Remove the method Find
Figure: Examples of instances generated by Pex to the LinkedList class.
Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space

12. Studied Tools - Korat - LinkedList
(a) LinkedList (b) LinkedList
instance with 2 instance with 5
elements elements
Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space

13. Studied Tools - Summary
Summary
Pex uses static analysis and is very efficient in discovering all the
possible execution paths in C# methods. Pex can also be used to
generate classes testcases, but the generated instances does not
keep the invariants of data structures.
On the other hand, Korat is the ideal tool to generate data
structures that meet the invariants.
Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space

14. Conclusion and Future work
Pex has proved to be a powerful tool regarding full coverage.
Korat is a very useful tool to generate complex data
structures.
A mix between the static analysis of Pex with Korat’s capability to
generate useful data structures is the path we will follow.
The study of pre- pos conditions inference using static analysis
[Moy 2009] will be useful to infer OCL rules.
Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space

15. Korat repOK method for LinkedList
1 public boolean repOK () {
2 if ( Head == null || Tail == null )
3 return size == 0;
4 if ( size == 1) return Head == Tail ;
5 if ( Head . Prev != null ) return false ;
6 if ( Tail . Next != null ) return false ;
7 LinkedListElement T last = Head ;
8 Set visited = new HashSet () ;
9 LinkedList workList = new LinkedList () ;
10 visited . add ( Head ) ;
11 workList . add ( Head ) ;
12 while (! workList . isEmpty () ) {
13 LinkedListElement T current = ( LinkedListElement T ) workList .
removeFirst () ;
14 if ( current . Next != null ) {
15 if (! visited . add ( current . Next ) )
16 return false ;
17 workList . add ( current . Next ) ;
18 if ( current . Next . Prev != current ) return false ;
19 last = current . Next ;
20 }
21 }
22 if ( last != Tail )
23 return false ;
24 return ( visited . size () == size ) ;
25 }
Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space