The document outlines the importance of automated tests in software development, providing motivations such as improved productivity, quality, and cost-effectiveness. It addresses common pretexts for not writing tests, highlights the benefits and pitfalls, and delineates best practices and types of tests necessary for effective software testing. Emphasizing that automated tests are essential for ongoing maintenance and refactoring, the document encourages developers to adopt a rigorous approach to testing.

1. A Case for Automated Tests
A Case for Automated Tests 1 / 31

2. Outline
1. Motivation
2. Pretexts not to write tests
3. Benefits and pitfalls
4. Best practices
5. Types of tests
A Case for Automated Tests 2 / 31

3. Motivation
1. Motivation
2. Pretexts not to write tests
3. Benefits and pitfalls
4. Best practices
5. Types of tests
A Case for Automated Tests 3 / 31

4. Motivation
Security
Writing software without automated tests...1
.
1
http://www.leonardscomic.com/comic/68/unit-testing/
A Case for Automated Tests 4 / 31

5. Motivation
Productivity
Constantly investing time on tests vs. having a single testing phase1
.
1
Andrew Hunt, David Thomas: Pragmatic Unit Testing in Java with JUnit
A Case for Automated Tests 5 / 31

6. Motivation
Money
The relative cost of fixing defects by product phase1
.
1
IBM Systems Sciences Institute
A Case for Automated Tests 6 / 31

7. Motivation
Productivity, Quality, Money
A Case for Automated Tests 7 / 31

8. Motivation
Why automated tests?
Automated tests
are reproducible (in contrast to manual tests)
provide fast feedback
all tests are run automatically for every code change
tests are documentation and provide examples
avoid ”works on my machine”
A Case for Automated Tests 8 / 31

9. Motivation
Why automated tests?
Automated tests are like a safety net
new features
updates
refactoring
A Case for Automated Tests 9 / 31

10. Motivation
Example: The Fear Factor
for (int i = 0; i < size; j++) {
// do stuff here
}
Is this a bug?
Or the reason that keeps the software running?
A Case for Automated Tests 10 / 31

11. Pretexts not to write tests
1. Motivation
2. Pretexts not to write tests
3. Benefits and pitfalls
4. Best practices
5. Types of tests
A Case for Automated Tests 11 / 31

12. Pretexts not to write tests
Time issues
I do not have the time
It takes too long
It’s not a factor in the project plan
Why you should still do it
It costs time to write good automated tests.
It saves time later on to have them.
Bug hunting
The more code your write without testing, the more paths you have to
check for errors.
A Case for Automated Tests 12 / 31

13. Pretexts not to write tests
Time issues
I do not have the time
It takes too long
It’s not a factor in the project plan
Why you should still do it
It costs time to write good automated tests.
It saves time later on to have them.
Bug hunting
The more code your write without testing, the more paths you have to
check for errors.
A Case for Automated Tests 12 / 31

14. Pretexts not to write tests
Time issues
I do not have the time
It takes too long
It’s not a factor in the project plan
Why you should still do it
It costs time to write good automated tests.
It saves time later on to have them.
Bug hunting
The more code your write without testing, the more paths you have to
check for errors.
A Case for Automated Tests 12 / 31

15. Pretexts not to write tests
Complexity issues
It is not possible
too many dependencies
initialising the required environment is cumbersome
the method does too much
Why you should still do it
If a method is hard to test it is probably also difficult to use and maintain
⇒ Refactoring (Collective Code Ownership)
A Case for Automated Tests 13 / 31

16. Pretexts not to write tests
Complexity issues
It is not possible
too many dependencies
initialising the required environment is cumbersome
the method does too much
Why you should still do it
If a method is hard to test it is probably also difficult to use and maintain
⇒ Refactoring (Collective Code Ownership)
A Case for Automated Tests 13 / 31

17. Pretexts not to write tests
Complexity Issues? Or Design Issues?
A Case for Automated Tests 14 / 31

18. Pretexts not to write tests
Maintenance
Test code needs maintenance
not flexible enough
refactoring
Why you should still do it
Test code is not throw-away code.
Test code may be more important than production code.
Safety. Again.
If you want to refactor, the essential precondition is having solid tests1.
1
Martin Fowler, Kent Beck: Refactoring: Improving the Design of Existing Code
A Case for Automated Tests 15 / 31

19. Pretexts not to write tests
Maintenance
Test code needs maintenance
not flexible enough
refactoring
Why you should still do it
Test code is not throw-away code.
Test code may be more important than production code.
Safety. Again.
If you want to refactor, the essential precondition is having solid tests1.
1
Martin Fowler, Kent Beck: Refactoring: Improving the Design of Existing Code
A Case for Automated Tests 15 / 31

20. Pretexts not to write tests
Maintenance
Test code needs maintenance
not flexible enough
refactoring
Why you should still do it
Test code is not throw-away code.
Test code may be more important than production code.
Safety. Again.
If you want to refactor, the essential precondition is having solid tests1.
1
Martin Fowler, Kent Beck: Refactoring: Improving the Design of Existing Code
A Case for Automated Tests 15 / 31

21. Benefits and pitfalls
1. Motivation
2. Pretexts not to write tests
3. Benefits and pitfalls
4. Best practices
5. Types of tests
A Case for Automated Tests 16 / 31

22. Benefits and pitfalls
Benefits
Automated tests are absolutely necessary for
Fearless Refactoring
Continuous Integration
Collective Code Ownership
A Case for Automated Tests 17 / 31

23. Benefits and pitfalls
Pitfalls
Tests are not a silver bullet.
A Case for Automated Tests 18 / 31

24. Benefits and pitfalls
Pitfalls
Automated tests
do not magically reveal all errors
do not fix badly written code
if carelessly written, may create a false illusion of safety.
A Case for Automated Tests 19 / 31

25. Benefits and pitfalls
Take away message
The novice says: ”I do not strive for 100% line coverage in tests; I only
write tests for the code that is important.”
The master says: ”If the code is not important, why is it there at all? I
will strive to test every line I write; if a line is not important, it should be
removed.”
1
Paul M. Jones, Live Coverage in Unit Tests
A Case for Automated Tests 20 / 31

26. Best practices
1. Motivation
2. Pretexts not to write tests
3. Benefits and pitfalls
4. Best practices
5. Types of tests
A Case for Automated Tests 21 / 31

27. Best practices
Basic properties of a good unit test
A unit test should
test exactly one component
test exactly one behaviour
be able to fail
logging statements are not sufficient.
use assertions. a lot.
Best practice
Avoid happy path testing. Try to break the code.
A Case for Automated Tests 22 / 31

28. Best practices
Basic properties of a good unit test
A unit test should
test exactly one component
test exactly one behaviour
be able to fail
logging statements are not sufficient.
use assertions. a lot.
Best practice
Avoid happy path testing. Try to break the code.
A Case for Automated Tests 22 / 31

29. Best practices
Implementing a new feature
Test
Implementation
all tests
ok?
fix
Commit/
Push
no
yes
A Case for Automated Tests 23 / 31

30. Best practices
Fixing a bug
Test that
reproduces
the bug
fix bug
all tests
ok?
fix
Commit/
Push
no
yes
A Case for Automated Tests 24 / 31

31. Types of tests
1. Motivation
2. Pretexts not to write tests
3. Benefits and pitfalls
4. Best practices
5. Types of tests
A Case for Automated Tests 25 / 31

32. Types of tests
Types of tests
unit tests
integration tests
system tests
acceptance tests
. . .
unit
cost
maintainance
coverage
Number of tests
integration
cost
maintainance
coverage
Number of tests
system
cost
maintainance
coverage
Number of tests
acceptance
cost
maintainance
coverage
Number of tests
A Case for Automated Tests 26 / 31

33. Types of tests
Unit Tests
Make sure that one particular unit of the
software does what it should.
Test my code, mock everything else.
unit
cost
maintainance
coverage
Number of tests
integration
cost
maintainance
coverage
Number of tests
system
cost
maintainance
coverage
Number of tests
acceptance
cost
maintainance
coverage
Number of tests
Keep in mind
Unit tests will not catch integration errors or broader system-level errors.
A Case for Automated Tests 27 / 31

34. Types of tests
Unit Tests
Make sure that one particular unit of the
software does what it should.
Test my code, mock everything else.
unit
cost
maintainance
coverage
Number of tests
integration
cost
maintainance
coverage
Number of tests
system
cost
maintainance
coverage
Number of tests
acceptance
cost
maintainance
coverage
Number of tests
Keep in mind
Unit tests will not catch integration errors or broader system-level errors.
A Case for Automated Tests 27 / 31

35. Types of tests
Integration Tests
Make sure that some modules of the
software work together properly.
my code + your code.
code + database.
unit
cost
maintainance
coverage
Number of tests
integration
cost
maintainance
coverage
Number of tests
system
cost
maintainance
coverage
Number of tests
acceptance
cost
maintainance
coverage
Number of tests
A Case for Automated Tests 28 / 31

36. Types of tests
System Tests
Make sure that everything works together
properly
my code + your code + database +
server + client + · · ·
unit
cost
maintainance
coverage
Number of tests
integration
cost
maintainance
coverage
Number of tests
system
cost
maintainance
coverage
Number of tests
acceptance
cost
maintainance
coverage
Number of tests
A Case for Automated Tests 29 / 31

37. Types of tests
Acceptance Tests
So far: did we build it right?
Acceptance test: did we build the right
thing?
unit
cost
maintainance
coverage
Number of tests
integration
cost
maintainance
coverage
Number of tests
system
cost
maintainance
coverage
Number of tests
acceptance
cost
maintainance
coverage
Number of tests
A Case for Automated Tests 30 / 31

38. So. Up to you...
A Case for Automated Tests 31 / 31