Kyiv Quality Assurance Day 2019 ГАННА КАПЛУН «Automatic test data generation» Телеграм канал: wwww.t.me/goqameetup Фейсбук сторінці: www.fb.com/goqaevent Сайт: www.kyiv.qaday.org

1. © Intellias 2016
Automatic test data generation
Or
Let’s make test design great again

2. Contents
Random Test Data Generation
Search-Based Test Data Generation
Genetic Algorithms and Programming
Test Sequence Generation
Tools

3. Work-Life Balance
Linkedin Facebook

4. Random Test Data Generation
Arbitrary inputs
Fast and easy
Very large test suites
Small programs only
Fuzzing

5. Search-Based Test Data Generation
Optimization problem
Very large input domain
Coverage threshold
No single correct answer

6. Genetic Algorithms and Programming
Fitness function
Populations and generations
Multiple targets in one run
Reproduction, crossover
and mutation

7. Test Sequence Generation
Combinatorial testing
Formal models
Generation rules
Dependency rules
Prioritization and weights

8. Generating Random Java Unit Tests with Randoop
Create a file myclasses.txt that lists the names of the classes under test:
java.util.Collections
java.util.TreeSet
and invoke Randoop as follows:
java -classpath $(RANDOOP_JAR) randoop.main.Main gentests --
classlist=myclasses.txt --time-limit=60
Randoop creates two different main test suites, in classes ErrorTest and
RegressionTest. Compile and run the tests.
All of the tests in ErrorTest will fail, and all of the tests in RegressionTest will pass.

9. Search-Based Tests with EvoSuite
1. What is the class under test?
2. What is the classpath where it can find the bytecode of the class under test and its
dependencies?
$EVOSUITE -class <ClassName> -projectCP <ClassPath>
Compile and run the tests.
Defining coverage type:
$EVOSUITE -class <ClassName> -projectCP <ClassPath> -criterion branch

10. Combinatorial Testing with PICT
Modelling:
BROWSER: IE, Firefox, Chrome, Opera
LANG: en, ru, ua
OS: win, linux, android
{BROWSER, LANG, OS} @ 1
IF [OS] = "linux" THEN [BROWSER] <> "IE";
Results:
IE ua win
Firefox en win
Opera ua linux
Chrome ru android

11. Useful Resources
TC generation tools:
http://mit.bme.hu/~micskeiz/pages/code_based_test_generation.html
Randoop:
https://randoop.github.io/randoop/
EvoSuite:
http://www.evosuite.org/
PICT:
https://github.com/Microsoft/pict

12. Q&A
Thanks for attention.
Any questions?