The document discusses various topics related to writing tests for software. It covers test structure and organization, what to test, using matchers, writing custom matchers, ignoring tests, handling exceptions, testing with time, testing collections, random values in tests, number of assertions per test, and decoupling in end-to-end tests. The presentation aims to provide best practices and anti-patterns to consider when writing and organizing test code.
1. Developer Tests
Things to Know
Vaidas Pilkauskas 2014
Vilnius JUG
vaidas.pilkauskas@gmail.com
2. me
My hobbies
● developer at Wix.com
● main language - Scala
● main professional interest - developer communities
If there is time left after my hobbies
● mountain bicycle rider, snowboarder
● consumer of rock music, contemporary art, etc
3. me - how to contact me
connect with me on LinkedIn http://lt.linkedin.
com/pub/vaidas-pilkauskas/8/77/863/
add me on G+ https://www.google.com/+VaidasPilkauskas
follow on Twitter @liucijus
4. “We couldn’t understand why people without technical
knowledge had to tell programmers “what” to do and,
furthermore, they had to supervise “how”
programmers did it.”
Cristian Rennella
http://qz.com/260846/why-our-startup-has-no-bosses-no-office-and-a-four-day-work-week/
5. What this talk is about
● Things we argue about during code reviews
● Things that took me time to understand and
prove that they are actually good way to go
● Small things we have no time to discuss
during big talks
6.
7. “Legacy Code is code without
Tests”
Michael Feathers
Working Effectively with Legacy Code
8. So what is test?
It is system’s exercise under predefined
conditions and then verification of an
expected outcome.
11. Test phases in code
Server server = new NotesServer(); // setup
Note note = new Note("test note"); // setup
Status status = server.add(note); // exercise
assertEquals(SUCCESS, status); // verify
server.shutdown(); // teardown
12. Start everything in one method
@Test
public void serverShouldAddNoteSuccessfully() {
Server server = new NotesServer(); // setup
Note note = new Note("test note"); // setup
Status status = server.add(note); // exercise
assertEquals(SUCCESS, status); // verify
server.shutdown(); // teardown
}
13. Refactor to lifecycle methods
@Before public void before() {
server = new NotesServer(); // setup
note = new Note("test note"); // setup
}
@Test public void serverShouldAddNoteSuccessfully() {
Status status = server.add(note); // exercise
assertEquals(SUCCESS, status); // verify
}
@After public void after() {
server.shutdown(); // teardown
}
24. Setup pollution - #5
● Setup hides details! Do not hide test preconditions.
25. Setting up “job” inside test method
@Before public void before() {
server = new NotesServer(); // setup
}
@Test public void serverShouldAddNoteSuccessfully() {
note = new Note("test note"); // setup
Status status = server.add(note); // exercise
assertEquals(SUCCESS, status); // verify
}
26. But then be more specific
@Test
public void serverShouldAddSingleLineNoteSuccesfully() {
// * set up which is actual for the current method
// * use scope specific name
Note singleLineNote = new Note("test note"); // setup
Status status = server.add(singleLineNote); // exercise
assertEquals(SUCCESS, status); // verify
}
27. Give good names to setup methods
@Before public void createNotesServer() {
server = new NotesServer();
}
28. Summary of test code organization
● DRY principle.
● Readability. BDD vs. DRY
● Consistency. Maintain the same style across
your codebase.
● Complexity. It may dictate the way you go.
29. Refactoring
Refactoring is about improving the design of
existing code. It is the process of changing a
software system in such a way that it does not
alter the external behavior of the code, yet
improves its internal structure.
Martin Fowler
Refactoring: Improving the Design of Existing Code
33. Test behaviour not methods
● Think of a contract
● And responsibilities
● Specify requirements as tests
34. Test behaviour not methods
● Think of a contract
● And responsibilities
● Specify requirements as tests
● Happens naturally when done in test-first
approach
36. Matchers
● Enhance readability
● Asserts on the right level of abstraction
● Encapsulate testing logic
● Reusable
● Detailed match error messages (do not
leave them out in your custom matchers!)
37. Matchers
● Enhance readability
● Asserts on the right level of abstraction
● Encapsulate testing logic
● Reusable
● Detailed match error messages (do not
leave them out in your custom matchers!)
38. Matcher libraries
● Hamcrest - standard matcher lib for JUnit
● AssertJ - fluent assertions (IDE friendly)
● Provides common matchers
● You can write your own custom matchers
43. Custom matchers
● Help communicate test intention
● Abstract assertion logic in case standard
matchers are not enough
● Are reusable and save time in large projects
● You may have a custom message to be
more specific about test failure
44. Custom matchers
● Help communicate test intention
● Abstract assertion logic in case standard
matchers are not enough
● Are reusable and save time in large projects
● You may have a custom message to be
more specific about test failure
45. Custom matchers
@Test
public void shouldHaveIsbnGenerated() {
Book book = new Book(1l, "5555", "A book");
assertThat(book, hasIsbn("1234"));
}
47. fail()
In some cases (e.g. testing exceptions) you
may want to force test to fail if some expected
situation does not happen
48. fail()
try {
// do stuff...
fail("Exception not thrown");
} catch(Exception e){
assertTrue(e.hasSomeFlag());
}
49. fail()
● Fundamentally not bad, but better use
matchers for expected failure
● Matchers help to clarify test intention
● Don’t forget - expected behaviour is an
opposite of a failing test
51. Anti-pattern: The Ugly Mirror
@Test
public void personToStringShouldIncludeNameAndSurname() {
Person person = new Person("Vilkas", "Pilkas");
String expected =
"Person[" + person.getName() + " " + person.getSurname() + "]"
assertEquals(expected, person.toString());
}
52. Anti-pattern: The Ugly Mirror
@Test
public void personToStringShouldIncludeNameAndSurname() {
Person person = new Person("Vilkas", "Pilkas");
String expected =
"Person[" + person.getName() + " " + person.getSurname() + "]"
assertEquals(expected, person.toString());
}
53. Anti-pattern: The Ugly Mirror
@Test
public void personToStringShouldIncludeNameAndSurname() {
Person person = new Person("Vilkas", "Pilkas");
assertEquals("Person[Vilkas Pilkas]", person.toString());
}
56. Why would you want to turn off the
test?
Well, because it fails… :)
57. Ignoring tests
● Always use ignore/pending API from your
test library (JUnit @Ignore)
58. Ignoring tests
● Always use ignore/pending API from your
test library (JUnit @Ignore)
● Do not comment out or false assert your test
59. Ignoring tests
● Always use ignore/pending API from your
test library (JUnit @Ignore)
● Do not comment out or false assert your test
● If you do not need a test - delete it
69. Problem
public class MyService {
...
public void process(LocalDate date) {
if (date.isBefore(LocalDate.now()) {
...
}
}
}
70. Testing with Time
● Design your system where time is a
collaborator
● Inject test specific time provider in your test
○ constant time
○ slow time
○ boundary cases time
71. Control time with Clock
public class MyService {
private Clock clock; // dependency inject
...
public void process(LocalDate date) {
if (date.isBefore(LocalDate.now(clock)) {
...
}
}
}
75. Collections
● Most of the time you want to assert on collection content
● Prefer exact content matching
76. Collections
● Most of the time you want to assert on collection content
● Prefer exact content matching
● Avoid incomplete assertions
77. Collections
● Most of the time you want to assert on collection content
● Prefer exact content matching
● Avoid incomplete assertions
● Do not sort just because it is easier to assert!
78. Collections
● Most of the time you want to assert on collection content
● Prefer exact content matching
● Avoid incomplete assertions
● Do not sort just because it is easier to assert!
● Multiple assertions are worse than single content
assertion
79. Collections
● Most of the time you want to assert on collection content
● Prefer exact content matching
● Avoid incomplete assertions
● Do not sort just because it is easier to assert!
● Multiple assertions are worse than single content
assertion
● Unless you want to say something important in your test!
80. Collections
● Most of the time you want to assert on collection content
● Prefer exact content matching
● Avoid incomplete assertions
● Do not sort just because it is easier to assert!
● Multiple assertions are worse than single content
assertion
● Unless you want to say something important in your test!
● Use matchers!
81. Collections
● Most of the time you want to assert on collection content
● Prefer exact content matching
● Avoid incomplete assertions
● Do not sort just because it is easier to assert!
● Multiple assertions are worse than single content
assertion
● Unless you want to say something important in your test!
● Use matchers!
84. Random values in tests
● Most of the time you do not want it
● Unless you depend on randomness a lot (eg.
password generation*)
*Thanks to Aleksandar Tomovski for a good example
85. Random values in tests
● Most of the time you do not want it
● Unless you depend on randomness a lot
● Use property based testing (which is also
hard)
86. Random values in tests
● Most of the time you do not want it
● Unless you depend on randomness a lot
● Use property based testing (which is also
hard)
● Do not make dummy values random
89. Generate Multiple Test Cases
● Quality over quantity
● Think of boundary cases, that you may want
to detect with random test
90. Generate Multiple Test Cases
● Quality over quantity
● Think of boundary cases, that you may want
to detect with random test
● Use parameterized tests
91. Generate Multiple Test Cases
● Quality over quantity
● Think of boundary cases, that you may want
to detect with random test
● Use parameterized tests
● Random is hard to repeat
92. Generate Multiple Test Cases
● Quality over quantity
● Think of boundary cases, that you may want
to detect with random test
● Use parameterized tests
● Random is hard to repeat
● Flickering tests
94. How many assertions per test?
● Unit test - one assertion per test. Must be
clear and readable
● Proper test should fail for exactly one reason
● End to end - best case one assertion per
test, but more are allowed
● Consider custom matchers
95. How many assertions per test?
● Unit test - one assertion per test. Must be
clear and readable
● Proper test should fail for exactly one
reason
● End to end - best case one assertion per
test, but more are allowed
● Consider custom matchers
97. What can be better in this test
[pseudocode]
@Test shouldRetrieveUserByLogin() {
String userJson = "{"username": "vaidas"}";
HttpRequest post = new Post("https://localhost:8080/users", userJson);
HttpResponse postResp = HttpClient().execute(post);
assertThat(postResp.status, is(200));
HttpRequest get = new Get("https://localhost:8080/users/vaidas");
HttpResponse getResp = HttpClient().execute(get);
User user = mapper.readValue(getResp, User.class);
assertThat(username, is("vaidas"));
}
99. Thoughts on end-to-end testing
● What is real e2e? REST vs. UI
● Testing by using system itself
● Vs. using low level DB driver to verify data
persistence
101. Comments in Test code
● Fundamentally good option to explain
complicated parts, but:
● better use good method naming
● custom matcher
● do less, so that intention is clear
● comments are not so bad in isolated well
named methods
102. Summary
● Context matters
● Tests are controversial topic
● And very opinionated