The document discusses the 'Code Defenders' project, which gamifies mutation testing to enhance teaching software testing at the University of Passau. It outlines how students take on roles of attackers and defenders through lab sessions and challenges, aiming to improve their skills in both areas. The findings suggest increased skill acquisition among students, with plans for future enhancements to the gameplay experience.

1. CODE DEFENDERSCODE DEFENDERS
GAMIFICATION OF MUTATION TESTINGGAMIFICATION OF MUTATION TESTING
Phil Werli - Student ResearcherPhil Werli - Student Researcher
Chair of Software Engineering II, University of PassauChair of Software Engineering II, University of Passau
Prof. Gordon FraserProf. Gordon Fraser
22. November 2019 - Madrid22. November 2019 - Madrid
1

2. TEACHING SOFTWARE TESTINGTEACHING SOFTWARE TESTING
2 . 1

3. CHALLENGECHALLENGE
Find an interesting way to teach Software Testing
2 . 2

4. MUTATION TESTINGMUTATION TESTING
1970’s — DeMillo (Georgia Tech), Lipton (Princeton), Sayward (Yale)
3 . 1

5. MUTATION TESTINGMUTATION TESTING
1970’s — DeMillo (Georgia Tech), Lipton (Princeton), Sayward (Yale)
Measure quality of existing tests
Modify source code: Mutants
Tests may detect and kill Mutants
3 . 1

6. EXAMPLEEXAMPLE
Original Code
int abs(int x) {
if (x >= 0) {
return x
} else {
return -x;
}
}
1
2
3
4
5
6
7
3 . 2

7. EXAMPLEEXAMPLE
Original Code
Test
int abs(int x) {
if (x >= 0) {
return x
} else {
return -x;
}
}
1
2
3
4
5
6
7
@Test
void testAbs() {
int res = abs(42);
assertEquals(42, res); // true
}
1
2
3
4
5
3 . 2

8. MUTATIONMUTATION
Mutant
if (x <= 0) {
int abs(int x) {1
2
return x3
} else {4
return -x;5
}6
}7
3 . 3

9. MUTATIONMUTATION
Mutant
Killing Test
if (x <= 0) {
int abs(int x) {1
2
return x3
} else {4
return -x;5
}6
}7
@Test
void testAbs() {
int res = abs(42);
assertEquals(42, res); // false -> mutant discovered
}
1
2
3
4
5
3 . 3

10. New Mutant
return -x + 1;
int abs(int x) {1
if (x >= 0) {2
return x3
} else {4
5
}6
}7
3 . 4

11. New Mutant
New Test
return -x + 1;
int abs(int x) {1
if (x >= 0) {2
return x3
} else {4
5
}6
}7
@Test
void testAbs() {
// ???
}
1
2
3
4
3 . 4

12. EQUIVALENCESEQUIVALENCES
Original Code
Equivalent Mutant
int abs(int x) {
if (x >= 0) {
return x
} else {
return -x;
}
}
1
2
3
4
5
6
7
int abs(int x) {
if (x >= 0) {
return x
} else {
return -x + 0;
}
}
1
2
3
4
5
6
7
3 . 5

13. CODE DEFENDERSCODE DEFENDERS
Attackers vs. Defenders
Attackers spawn Mutants
Defenders defend with Tests
4 . 1

14. LIVE DEMOLIVE DEMO
Enough talking...Enough talking...
4 . 2

15. SOFTWARE TESTING COURSE @ UNI PASSAUSOFTWARE TESTING COURSE @ UNI PASSAU
2h Lecture
1h Exercise
2h Lab Session
around 120 students
5 . 1

16. Lab SessionsLab Sessions
1 session per week per student
Teams of 3-5 students
Students play both roles
Increasing difficulty
Count towards grade
5 . 2

17. Four Challenges
Classroom Management Games Management
Course Management Assessment
5 . 3

18. LIVE DEMOLIVE DEMO
Admin interfaceAdmin interface
5 . 4

19. COURSE DATA & STUDYCOURSE DATA & STUDY
Data per game
Data per student
Study with data from our course
6 . 1

20. IMPROVEMENT THROUGHOUT SEMESTERIMPROVEMENT THROUGHOUT SEMESTER
6 . 2

21. GAME ACTIVITY VS. EXAM GRADESGAME ACTIVITY VS. EXAM GRADES
6 . 3

22. ANONYMOUS SURVEYANONYMOUS SURVEY
84% learned useful skills as defenders
60% learned useful skills as attackers
=> Make people defend more=> Make people defend more
6 . 4

23. FUTURE CHALLENGESFUTURE CHALLENGES
Communication mechanism to foster collaboration
Many opportunities to spice up gameplay
Integrate more technologies & features
7 . 1

24. PUBLICLY AVAILABLEPUBLICLY AVAILABLE
code-defenders.org
github.com/CodeDefenders
7 . 2

25. Overview
Play onlinePlay online
Source CodeSource Code
https://code-defenders.orghttps://code-defenders.org
github.com/CodeDefendersgithub.com/CodeDefenders
7 . 3

26. CHAIR & SPEAKER CONTACTCHAIR & SPEAKER CONTACT
Presentation created usingPresentation created using
Chair of Software Engineering II
github.com/werli
reveal.jsreveal.js
7 . 4