Testing Zen
Improve software quality
and one's quality of life
with fuzz tests, code coverage,
and a fast in-memory database.
Thomas Müller
Day Software AG
Presentation 6640

2
Agenda
• Introduction
• Why Test?
• Test Strategies
1 2 3
Fuz z Te s t C o ve r ag e Dat a b a s e

3
Introduction
Thomas Mueller
Software Engineer
http://www.h2database.com
http://www.day.com
http://jackrabbit.apache.org

3
Introduction
Thomas Mueller
Software Engineer
http://www.h2database.com
http://www.day.com
http://jackrabbit.apache.org

3
Introduction
Thomas Mueller
Software Engineer
http://www.h2database.com
http://www.day.com
http://jackrabbit.apache.org

3
Introduction
Thomas Mueller
Software Engineer
http://www.h2database.com
http://www.day.com
http://jackrabbit.apache.org

4
Why Automated Tests?

4
Why Automated Tests?
1. Less Bugs

4
Why Automated Tests?
1. Less Bugs
2. Can Refactor

4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code

4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work

4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs

4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
6. Short Release Cycle

4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
6. Short Release Cycle
7. Less/No Branching

4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
6. Short Release Cycle
7. Less/No Branching
8. Predictable Work

4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
6. Short Release Cycle
7. Less/No Branching
8. Predictable Work
9. Go Home Early

4
Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
6. Short Release Cycle
7. Less/No Branching
8. Predictable Work
9. Go Home Early
10. Beer!

5
Testing Strategies

5
Testing Strategies
1 2 3
Fuz z Te s t C o ve r ag e Da t a b a s e

6

6
1
Fu zz Te st

7
Fuzz Testing
• Random data, operations
• Low cost, high benefit
• Find bugs humans don’t
• Repeatable
• Tune with code coverage

8
Fuzz Testing Example
interface DataService {
void create(int k, String v);
String read(int k);
void delete(int k);
}

8
Fuzz Testing Example
interface DataService {
void create(int k, String v);
String read(int k);
void delete(int k);
}
byte[] b = new byte[100];
random.nextBytes(b);
data = new String(b,
"ISO-8859-1");

8
Fuzz Testing Example
interface DataService { Random r = new Random(123);
void create(int k, String v); for (int i = 0; i < 10000; i++) {
String read(int k); int k = r.nextInt(100);
void delete(int k); switch (r.nextInt(3)) {
} case 0:
ds.create(k, randomString(r));
break;
case 1:
ds.read(k);
byte[] b = new byte[100]; break;
random.nextBytes(b); case 2:
data = new String(b, ds.delete(k);
"ISO-8859-1"); break;
}
}

9

9
2
C o ve ra ge

10
Code Coverage
• Measure what is tested
• Covered classes,
methods, blocks, lines
• Find untested, dead code
• Tools: Emma and others

11
Code Coverage Example

11
Code Coverage Example
tested
untested
partially

12
Code Coverage Example

13

13
3
D at a base

14
Database Testing
• Application uses a big
database (Oracle,...)
• Install pain
• Memory usage
• License
• Slow

15
Java SQL Database

15
Java SQL Database

15
Java SQL Database
Size
In-Memory
Performance
Compatible

15
Java SQL Database
Size 1 MB
In-Memory ++
Performance ++
Compatible +

15
Java SQL Database
Size 1 MB 2 MB
In-Memory ++ -
Performance ++ -
Compatible + -

15
Java SQL Database
Size 1 MB 2 MB 1 MB
In-Memory ++ - ++
Performance ++ - ++
Compatible + - ++

Images are Creative Commons licensed:
Thomas Mueller Test Well
Software Engineer http://www.flickr.com/photos/cobalt/409924867
Beer
http://www.flickr.com/photos/nickwheeleroz/2178156484
http://www.h2database.com
Boring
http://www.day.com http://www.flickr.com/photos/phoenixdailyphoto/1467681879
http://jackrabbit.apache.org Dice
http://www.flickr.com/photos/tim_ellis/402207052
Dice (black)
http://www.flickr.com/photos/mkamp/2478311790
Static Analysis:
Checkstyle, FindBugs, PMD
Continuous Integration:
Hudson, Apache Continuum
Continuous Test Runner:
Infinitest