1. Testing Zen
Improve software quality
and one's quality of life
with fuzz tests, code coverage,
and a fast in-memory database.
Thomas Müller
Day Software AG
Presentation 6640

2. 2
Agenda
• Introduction
• Why Test?
• Test Strategies
1 2 3
Fuz z Te s t C o ve r ag e Dat a b a s e

3. 3
Introduction
Thomas Mueller
Software Engineer
http://www.h2database.com
http://www.day.com
http://jackrabbit.apache.org

4. 3
Introduction
Thomas Mueller
Software Engineer
http://www.h2database.com
http://www.day.com
http://jackrabbit.apache.org

5. 3
Introduction
Thomas Mueller
Software Engineer
http://www.h2database.com
http://www.day.com
http://jackrabbit.apache.org

6. 3
Introduction
Thomas Mueller
Software Engineer
http://www.h2database.com
http://www.day.com
http://jackrabbit.apache.org

7. 4
Why Automated Tests?

8. 4
Why Automated Tests?
1. Less Bugs

9. 4
Why Automated Tests?
1. Less Bugs
2. Can Refactor

10. 4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code

11. 4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work

12. 4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs

13. 4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
6. Short Release Cycle

14. 4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
6. Short Release Cycle
7. Less/No Branching

15. 4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
6. Short Release Cycle
7. Less/No Branching
8. Predictable Work

16. 4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
6. Short Release Cycle
7. Less/No Branching
8. Predictable Work
9. Go Home Early

17. 4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
6. Short Release Cycle
7. Less/No Branching
8. Predictable Work
9. Go Home Early
10. Beer!

18. 5
Testing Strategies

19. 5
Testing Strategies
1 2 3
Fuz z Te s t C o ve r ag e Da t a b a s e

20. 6

21. 6
1
Fu zz Te st

22. 7
Fuzz Testing
• Random data, operations
• Low cost, high benefit
• Find bugs humans don’t
• Repeatable
• Tune with code coverage

23. 8
Fuzz Testing Example
interface DataService {
void create(int k, String v);
String read(int k);
void delete(int k);
}

24. 8
Fuzz Testing Example
interface DataService {
void create(int k, String v);
String read(int k);
void delete(int k);
}
byte[] b = new byte[100];
random.nextBytes(b);
data = new String(b,
"ISO-8859-1");

25. 8
Fuzz Testing Example
interface DataService { Random r = new Random(123);
void create(int k, String v); for (int i = 0; i < 10000; i++) {
String read(int k); int k = r.nextInt(100);
void delete(int k); switch (r.nextInt(3)) {
} case 0:
ds.create(k, randomString(r));
break;
case 1:
ds.read(k);
byte[] b = new byte[100]; break;
random.nextBytes(b); case 2:
data = new String(b, ds.delete(k);
"ISO-8859-1"); break;
}
}

26. 9

27. 9
2
C o ve ra ge

28. 10
Code Coverage
• Measure what is tested
• Covered classes,
methods, blocks, lines
• Find untested, dead code
• Tools: Emma and others

29. 11
Code Coverage Example

30. 11
Code Coverage Example
tested
untested
partially

31. 12
Code Coverage Example

32. 13

33. 13
3
D at a base

34. 14
Database Testing
• Application uses a big
database (Oracle,...)
• Install pain
• Memory usage
• License
• Slow

35. 15
Java SQL Database

36. 15
Java SQL Database

37. 15
Java SQL Database
Size
In-Memory
Performance
Compatible

38. 15
Java SQL Database
Size 1 MB
In-Memory ++
Performance ++
Compatible +

39. 15
Java SQL Database
Size 1 MB 2 MB
In-Memory ++ -
Performance ++ -
Compatible + -

40. 15
Java SQL Database
Size 1 MB 2 MB 1 MB
In-Memory ++ - ++
Performance ++ - ++
Compatible + - ++

42. Images are Creative Commons licensed:
Thomas Mueller Test Well
Software Engineer http://www.flickr.com/photos/cobalt/409924867
Beer
http://www.flickr.com/photos/nickwheeleroz/2178156484
http://www.h2database.com
Boring
http://www.day.com http://www.flickr.com/photos/phoenixdailyphoto/1467681879
http://jackrabbit.apache.org Dice
http://www.flickr.com/photos/tim_ellis/402207052
Dice (black)
http://www.flickr.com/photos/mkamp/2478311790
Static Analysis:
Checkstyle, FindBugs, PMD
Continuous Integration:
Hudson, Apache Continuum
Continuous Test Runner:
Infinitest