Testing Zen
Improve software quality
and one's quality of life
with fuzz tests, code coverage,
and a fast in-memory databa...
2



Agenda
• Introduction
• Why Test?



• Test Strategies

      1               2              3
  Fuz z Te s t   C o v...
3




Introduction

 Thomas Mueller
 Software Engineer

 http://www.h2database.com
 http://www.day.com
 http://jackrabbit....
3




Introduction

 Thomas Mueller
 Software Engineer

 http://www.h2database.com
 http://www.day.com
 http://jackrabbit....
3




Introduction

 Thomas Mueller
 Software Engineer

 http://www.h2database.com
 http://www.day.com
 http://jackrabbit....
3




Introduction

 Thomas Mueller
 Software Engineer

 http://www.h2database.com
 http://www.day.com
 http://jackrabbit....
4




Why Automated Tests?
4




Why Automated Tests?
1. Less Bugs
4




Why Automated Tests?
1. Less Bugs
2. Can Refactor
4




Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4




Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
4




Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
4




Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
6. Short ...
4




Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
6. Short ...
4




Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
6. Short ...
4




Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
6. Short ...
4




Why Automated Tests?
1. Less Bugs
2. Can Refactor
3. Nice Code
4. Nice to Work
5. Tests are Samples & Docs
6. Short ...
5




Testing Strategies
5




Testing Strategies




        1                 2               3
    Fuz z Te s t     C o ve r ag e   Da t a b a s...
6
6




    1
Fu zz Te st
7




Fuzz Testing
•   Random data, operations
•   Low cost, high benefit
•   Find bugs humans don’t
•   Repeatable
•   Tu...
8




Fuzz Testing Example
 interface DataService {
    void create(int k, String v);
    String read(int k);
    void del...
8




Fuzz Testing Example
 interface DataService {
    void create(int k, String v);
    String read(int k);
    void del...
8




Fuzz Testing Example
 interface DataService {            Random r = new Random(123);
    void create(int k, String v...
9
9




     2
C o ve ra ge
10




Code Coverage
• Measure what is tested
• Covered classes,
  methods, blocks, lines
• Find untested, dead code
• Too...
11




Code Coverage Example
11




Code Coverage Example



                        tested
                        untested



                       ...
12




Code Coverage Example
13
13




     3
D at a base
14




Database Testing
• Application uses a big
    database (Oracle,...)
•   Install pain
•   Memory usage
•   License
•...
15




Java SQL Database
15




Java SQL Database
15




Java SQL Database




Size
In-Memory
Performance
Compatible
15




Java SQL Database




Size          1 MB
In-Memory      ++
Performance    ++
Compatible     +
15




Java SQL Database




Size          1 MB   2 MB
In-Memory      ++     -
Performance    ++     -
Compatible     +   ...
15




Java SQL Database




Size          1 MB   2 MB   1 MB
In-Memory      ++     -      ++
Performance    ++     -     ...
Images are Creative Commons licensed:
Thomas Mueller                 Test Well
Software Engineer              http://www.f...
Testing Zen
Upcoming SlideShare
Loading in...5
×

Testing Zen

746

Published on

Published in: Spiritual, Technology
1 Comment
1 Like
Statistics
Notes
No Downloads
Views
Total Views
746
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
8
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Testing Zen

  1. 1. Testing Zen Improve software quality and one's quality of life with fuzz tests, code coverage, and a fast in-memory database. Thomas Müller Day Software AG Presentation 6640
  2. 2. 2 Agenda • Introduction • Why Test? • Test Strategies 1 2 3 Fuz z Te s t C o ve r ag e Dat a b a s e
  3. 3. 3 Introduction Thomas Mueller Software Engineer http://www.h2database.com http://www.day.com http://jackrabbit.apache.org
  4. 4. 3 Introduction Thomas Mueller Software Engineer http://www.h2database.com http://www.day.com http://jackrabbit.apache.org
  5. 5. 3 Introduction Thomas Mueller Software Engineer http://www.h2database.com http://www.day.com http://jackrabbit.apache.org
  6. 6. 3 Introduction Thomas Mueller Software Engineer http://www.h2database.com http://www.day.com http://jackrabbit.apache.org
  7. 7. 4 Why Automated Tests?
  8. 8. 4 Why Automated Tests? 1. Less Bugs
  9. 9. 4 Why Automated Tests? 1. Less Bugs 2. Can Refactor
  10. 10. 4 Why Automated Tests? 1. Less Bugs 2. Can Refactor 3. Nice Code
  11. 11. 4 Why Automated Tests? 1. Less Bugs 2. Can Refactor 3. Nice Code 4. Nice to Work
  12. 12. 4 Why Automated Tests? 1. Less Bugs 2. Can Refactor 3. Nice Code 4. Nice to Work 5. Tests are Samples & Docs
  13. 13. 4 Why Automated Tests? 1. Less Bugs 2. Can Refactor 3. Nice Code 4. Nice to Work 5. Tests are Samples & Docs 6. Short Release Cycle
  14. 14. 4 Why Automated Tests? 1. Less Bugs 2. Can Refactor 3. Nice Code 4. Nice to Work 5. Tests are Samples & Docs 6. Short Release Cycle 7. Less/No Branching
  15. 15. 4 Why Automated Tests? 1. Less Bugs 2. Can Refactor 3. Nice Code 4. Nice to Work 5. Tests are Samples & Docs 6. Short Release Cycle 7. Less/No Branching 8. Predictable Work
  16. 16. 4 Why Automated Tests? 1. Less Bugs 2. Can Refactor 3. Nice Code 4. Nice to Work 5. Tests are Samples & Docs 6. Short Release Cycle 7. Less/No Branching 8. Predictable Work 9. Go Home Early
  17. 17. 4 Why Automated Tests? 1. Less Bugs 2. Can Refactor 3. Nice Code 4. Nice to Work 5. Tests are Samples & Docs 6. Short Release Cycle 7. Less/No Branching 8. Predictable Work 9. Go Home Early 10. Beer!
  18. 18. 5 Testing Strategies
  19. 19. 5 Testing Strategies 1 2 3 Fuz z Te s t C o ve r ag e Da t a b a s e
  20. 20. 6
  21. 21. 6 1 Fu zz Te st
  22. 22. 7 Fuzz Testing • Random data, operations • Low cost, high benefit • Find bugs humans don’t • Repeatable • Tune with code coverage
  23. 23. 8 Fuzz Testing Example interface DataService { void create(int k, String v); String read(int k); void delete(int k); }
  24. 24. 8 Fuzz Testing Example interface DataService { void create(int k, String v); String read(int k); void delete(int k); } byte[] b = new byte[100]; random.nextBytes(b); data = new String(b, "ISO-8859-1");
  25. 25. 8 Fuzz Testing Example interface DataService { Random r = new Random(123); void create(int k, String v); for (int i = 0; i < 10000; i++) { String read(int k); int k = r.nextInt(100); void delete(int k); switch (r.nextInt(3)) { } case 0: ds.create(k, randomString(r)); break; case 1: ds.read(k); byte[] b = new byte[100]; break; random.nextBytes(b); case 2: data = new String(b, ds.delete(k); "ISO-8859-1"); break; } }
  26. 26. 9
  27. 27. 9 2 C o ve ra ge
  28. 28. 10 Code Coverage • Measure what is tested • Covered classes, methods, blocks, lines • Find untested, dead code • Tools: Emma and others
  29. 29. 11 Code Coverage Example
  30. 30. 11 Code Coverage Example tested untested partially
  31. 31. 12 Code Coverage Example
  32. 32. 13
  33. 33. 13 3 D at a base
  34. 34. 14 Database Testing • Application uses a big database (Oracle,...) • Install pain • Memory usage • License • Slow
  35. 35. 15 Java SQL Database
  36. 36. 15 Java SQL Database
  37. 37. 15 Java SQL Database Size In-Memory Performance Compatible
  38. 38. 15 Java SQL Database Size 1 MB In-Memory ++ Performance ++ Compatible +
  39. 39. 15 Java SQL Database Size 1 MB 2 MB In-Memory ++ - Performance ++ - Compatible + -
  40. 40. 15 Java SQL Database Size 1 MB 2 MB 1 MB In-Memory ++ - ++ Performance ++ - ++ Compatible + - ++
  41. 41. Images are Creative Commons licensed: Thomas Mueller Test Well Software Engineer http://www.flickr.com/photos/cobalt/409924867 Beer http://www.flickr.com/photos/nickwheeleroz/2178156484 http://www.h2database.com Boring http://www.day.com http://www.flickr.com/photos/phoenixdailyphoto/1467681879 http://jackrabbit.apache.org Dice http://www.flickr.com/photos/tim_ellis/402207052 Dice (black) http://www.flickr.com/photos/mkamp/2478311790 Static Analysis: Checkstyle, FindBugs, PMD Continuous Integration: Hudson, Apache Continuum Continuous Test Runner: Infinitest
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×