This document provides an overview of mutation testing and the Pitest tool. It defines mutation testing as seeding artificial defects into source code to check if tests detect them. It describes various mutators that Pitest uses to alter code, such as changing relational operators or removing method calls. The document also outlines how to set up Pitest in a Maven project and review the test coverage report. It concludes that mutation testing can find bugs and redundant code but requires significant time to run all mutants. Not all mutants need to be killed to ensure quality.

1. Mutation testing
with Pitest

2. Agenda
1) Mutation testing - definition
2) Mutators
3) Setup guide of Pitest in Maven
4) Run mutation tests
5) Pitest Coverage Report
6) Conclusions
7) References

3. Mutation testing
Definition
1

4. Definition
Mutation testing was originally proposed by Richard Lipton as a
student in 1971 and first developed and published by DeMillo,
Lipton and Sayward. The first implementation of a mutation
testing tool has been created by Timothy Budd as part of his PhD
work (titled Mutation Analysis) in 1980 from Yale University. [6]

5. Definition
Mutation testing is used to design new software tests and
evaluate the quality of existing software tests. Basically the idea
is about seeding artificial defects (mutations) into a source code
and checks whether your test suite finds them.
If you change the source code
▪ your test fails then mutation is killed - It’s good :)
▪ your test passes then mutation survived - It’s bad :(

7. Mutators
2

8. Mutators
Run by default by Pitest :
▪ INCREMENTS_MUTATOR
▪ CONDITIONALS_BOUNDARY_MUTATOR (e.g.)
▪ RETURN_VALS_MUTATOR
▪ VOID_METHOD_CALL_MUTATOR (e.g.)
▪ INVERT_NEGS_MUTATOR
▪ MATH_MUTATOR
▪ NEGATE_CONDITIONALS_MUTATOR (e.g.)

9. Conditionals Boundary
Mutator
if (a < b) {
//some code
}
mutated into
if (a <= b) {
//some code
}
Original Mutated
< <=
<= <
> >=
>= >

10. Negate Conditionals Mutator
if (a == b) {
//some code
}
mutated into
if (a != b) {
//some code
}
Original Mutated
== !=
!= ==
<= >
>= <
< >=
> <=

11. Void method call mutator
public void myMethod(int a) {
//some code
}
public int rootMethod(int x) {
myMethod(x);
return x;
}
mutated into
public void myMethod(int a) {
//some code
}
public int rootMethod(int x) {
return x;
}

12. Mutators
Other mutators disabled by default :
▪ CONSTRUCTOR_CALLS - replaces constructor call with null
▪ INLINE_CONSTS - assigns value to non-final variable
▪ NON_VOID_METHOD_CALLS - removes calls to non void methods
▪ REMOVE_CONDITIONALS - removes all conditionals statements
▪ EXPERIMENTAL_MEMBER_VARIABLE - removes assignments to
member variables
▪ EXPERIMENTAL_SWITCH - changes switch statement by replacing
the default label (wherever it is used) with first label. All the other
labels are replaced by the default one

13. Setup guide of
Pitest in Maven3

15. Run mutation
tests
4

18. Pitest
Coverage Report
6

20. Conclusions
7

21. Conclusions
Mutation testing seems powerful and research indicates that
mutation score is a better predictor of real fault detection rate
than code coverage [2]. However, it has not yet received
widespread popularity.
Creating mutations and executing tests against those mutations
is not a lightweight process and can take quite a lot of time.
In addition to finding bugs, mutation testing is a great way to
find redundant code thus making your code cleaner!

22. Conclusions
Should all mutants be killed ? No, but it depends.
try {
connection.close();
} catch(SQLException e){
doNext("Call DevOps & create JIRA task for Dev :)");
}
To get 100% line coverage here would mean adding a test scenario that throws an
error on connection.close() and verify the call to doNext(..). If you haven’t got it
covered, PIT will remove the call to ‘doNext’ and complain in turn that you didn’t
kill that mutant. By all means do, if your time and budget are limitless.
Remember! Mutant is only an instance in which an artificial change to your code
was not detected by your tests.

23. References
8

24. References
1) www.pitest.org
2) Are Mutants a Valid Substitute for Real Faults in Software
Testing? https://people.cs.umass.
edu/~rjust/publ/mutants_real_faults_fse_2014.pdf
3) https://blog.codecentric.de/en/2016/02/sensible-mutation-
testing-dont-go-killing-spree-2/
4) http://www.hascode.com/2015/05/mutation-testing-with-
pitest-and-maven/
5) https://gofore.com/find-bugs-mutation-testing/
6) https://en.wikipedia.org/wiki/Mutation_testing

25. THANKS!
Any questions ?
Thank you http://www.slidescarnival.com for layout