Developer Tests
Things to Know
Vaidas Pilkauskas 2014
Kaunas JUG
vaidas.pilkauskas@gmail.com

“We couldn’t understand why people without technical
knowledge had to tell programmers “what” to do and,
furthermore, they had to supervise “how”
programmers did it.”
Cristian Rennella
http://qz.com/260846/why-our-startup-has-no-bosses-no-office-and-a-four-day-work-week/

What I’m going to talk about
● Things we argue about during code reviews
● Things that took me time to understand and
prove that they are actually good way to go
● Small things we have no time to discuss
during big talks

“Legacy Code is code without
Tests”
Michael Feathers
Working Effectively with Legacy Code

So what is test?
It is system’s exercise under predefined
conditions and then verification of an
expected outcome.

Thing #1
Test phases

Test phases
1. Set up
2. Exercise
3. Verify
4. Tear down

Test phases
@Test public void serverShouldExecuteJobSuccessfully() {
Server server = new Server(); // set up
Job job = new Job(); // set up
Status status = server.execute(job); // exercise
assertEquals(SUCCESS, status); // verify
server.shutdown(); // tear down
}

@Before public void before() {
server = new Server();
}
@Test public void serverShouldExecuteJobSuccessfully() {
Job job = new Job(); // set up
Status status = server.execute(job); // exercise
assertEquals(SUCCESS, status); // verify
server.shutdown(); // tear down
}

@Before public void before() {
server = new Server();
Job job = new Job();
}
@Test public void serverShouldExecuteJobSuccessfully() {
Status status = server.execute(job); // exercise
assertEquals(SUCCESS, status); // verify
server.shutdown(); // tear down
}

@Test public void serverShouldQueueJobWithFutureDate() {
// * set up which is actual for the current method
// * use scope specific name
Job futureJob = new Job(futureDate()); // set up
Status status = server.execute(futureJob); // exercise
assertEquals(SUCCESS, status); // verify
server.shutdown(); // tear down
}

@Before public void before() {
server = new Server();
Job job = new Job();
}
@Test public void serverShouldExecuteJobSuccessfully() {
Status status = server.execute(job); // exercise
assertEquals(SUCCESS, status); // verify
}
@After public void after() {
server.shutdown(); // tear down
}

@Before ..
@Test public void serverShouldExecuteJobSuccessfully() {
// * no need to name intermediate var, but
// * may hide return meaning of server.execute()
// execute & verify
assertEquals(SUCCESS, server.execute(job));
}
@After ..

Set up
● DRY principle

Set up
● DRY principle
● Readability

Set up
● DRY principle
● Readability
● Consistency

Set up
● DRY principle
● Readability
● Consistency
● Complexity

Refactoring
Refactoring is about improving the design of
existing code. It is the process of changing a
software system in such a way that it does not
alter the external behavior of the code, yet
improves its internal structure.
Martin Fowler
Refactoring: Improving the Design of Existing Code

Thing #2
What do we test?

Test behaviour not methods
● Think of a contract

Test behaviour not methods
● Think of a contract
● And responsibilities

Test behaviour not methods
● Think of a contract
● And responsibilities
● Specify requirements as tests

Test behaviour not methods
● Think of a contract
● And responsibilities
● Specify requirements as tests
● Happens naturally when done in test-first
approach

Thing #3
Matchers

Matchers
● Enhanced readability
● Assertions on the right level of abstraction
● Encapsulate testing logic
● Reusable
● Detailed match error messages (do not
leave them out in your custom matchers!)

Matcher libraries
● Hamcrest - standard matcher lib for JUnit
● AssertJ - fluent assertions (IDE friendly)
● Bring common matchers for you to use
● Write your own custom matchers

Hamcrest
assertThat(theBiscuit, equalTo(myBiscuit));
assertThat(theBiscuit, is(equalTo(myBiscuit)));
assertThat(theBiscuit, is(myBiscuit));

AssertJ
assertThat(frodo.getName()).isEqualTo("Frodo");
assertThat(frodo).isNotEqualTo(sauron)
.isIn(fellowshipOfTheRing);
assertThat(sauron).isNotIn(fellowshipOfTheRing);

Thing #4
Custom matchers

Custom matchers
Are matchers we develop specifically for our
projects

Custom matchers
● Help communicate test intention
● Abstract assertion logic in case standard
matchers are not enough
● Are reusable and save time in large projects
● You may have custom message to be more
specific about test failure

Custom matchers
@Test
public void testBookIsbn() {
Book book = new Book(1l, "5555", "A book");
assertThat(book, hasIsbn("1234"));
}

Thing #5
Failing test

fail()
In some cases like testing exceptions you may
want to force test to fail if some expected
situation does not happen

fail()
try{
// do stuff...
fail("Exception not thrown");
}catch(Exception e){
assertTrue(e.hasSomeFlag());
}

fail()
● Fundamentally not bad, but better use
matchers for expected failure
● Matchers help to clarify test intention
● Don’t forget - expected behaviour is an
opposite of a failing test

Thing #6
Anti-pattern: The Ugly Mirror

Anti-pattern: The Ugly Mirror
@Test
public void personToStringShouldIncludeNameAndSurname() {
Person person = new Person("Vilkas", "Pilkas");
String expected =
"Person[" + person.getName() + " " + person.getSurname() + "]"
assertEquals(expected, person.toString());
}

Anti-pattern: The Ugly Mirror
@Test
public void personToStringShouldIncludeNameAndSurname() {
Person person = new Person("Vilkas", "Pilkas");
String expected =
"Person[" + person.getName() + " " + person.getSurname() + "]"
assertEquals(expected, person.toString());
}

Anti-pattern: The Ugly Mirror
@Test
public void personToStringShouldIncludeNameAndSurname() {
Person person = new Person("Vilkas", "Pilkas");
assertEquals("Person[Vilkas Pilkas]", person.toString());
}

Thing #7
How to turn off the test?

Why would you want to turn off the
test?
● Well, because it fails… :)

Ignoring tests
● Always use ignore/pending API from your
test library (JUnit @Ignore)

Ignoring tests
● Always use ignore/pending API from your
test library (JUnit @Ignore)
● Do not comment out or false assert your test

Ignoring tests
● Always use ignore/pending API from your
test library (JUnit @Ignore)
● Do not comment out or false assert your test
● If you do not need a test - delete it

Thing #8
What to do with exceptions?

Exceptions
● If you can, use matchers instead of
○ @Test(expected=?)

JUnit expected exception
@Test(expected=IndexOutOfBoundsException.class)
public void shouldThrowIndexOutOfBoundsException() {
ArrayList emptyList = new ArrayList();
Object o = emptyList.get(0);
}
//matcher in Specs2 (Scala)
server.process(None) must throwA[NothingToProccess]

Exceptions
● If you can, use matchers instead of
○ @Test(expected=?)
○ try-catch approach

try and catch
public void shouldThrowIndexOutOfBoundsException() {
ArrayList emptyList = new ArrayList();
try {
Object o = emptyList.get(0);
fail("Should throw IndexOutOfBoundsException");
} catch(IndexOutOfBoundsException e)){
//consider asserting message!
}
}

Exceptions
● If you can, use matchers instead of
○ @Test(expected=?)
○ try-catch approach
● catch-exception lib

catch-exception lib
List myList = new ArrayList();
catchException(myList).get(1);
assertThat(caughtException(),
allOf(
is(IndexOutOfBoundsException.class),
hasMessage("Index: 1, Size: 0"),
hasNoCause()
)
);

Exceptions
● If you can, use matchers instead of
○ @Test(expected=?)
○ try-catch approach
● catch-exception lib
● What about ExpectedException Rule?
○ My personal opinion - not that intuitive
○ breaks arrange/act/assert flow

ExpectedException rule
@Rule public ExpectedException exception = ExpectedException.none();
@Test
public void testExpectedException() {
exception.expect(IllegalArgumentException.class);
exception.expectMessage(containsString('Invalid age'));
new Person('Vilkas', -1);
}
//Person constructor
public Person(String name, int age) {
if (age <= 0) throw new IllegalArgumentException('Invalid age:' + age);
// ...
}

Thing #9
How to test asynchronous code?

Asynchronous code
● Do not Thread.sleep - makes test slow

Asynchronous code
● Do not Thread.sleep - makes test slow
● Use Awaitility, or similar DSL for
synchronizing asynchronous operations

Awaitility (Java 8 example)
@Test
public void shouldPersistNewUser() {
publish(new CreateUserCommand("Vilkas Pilkas"));
await().until(userRepo::size, is(1));
//how long to await? (Default is 10 seconds)
await().until(userRepo::isNotEmpty);
}

Asynchronous code
● Do not Thread.sleep - makes test slow
● Use Awaitility, or similar DSL for
synchronizing asynchronous operations
● Use reasonable await time to avoid flaky
tests

Thing #10
Testing with time

Problem
public class MyService {
...
public void process(LocalDate date) {
if (date.isBefore(LocalDate.now()) {
...
}
}
}

Testing with Time
● Design your system where time is a
collaborator
● Inject test specific time provider in your test
○ constant time
○ slow time
○ boundary cases time

Control time with Clock
public class MyService {
private Clock clock; // dependency inject
...
public void process(LocalDate date) {
if (date.isBefore(LocalDate.now(clock)) {
...
}
}
}

Thing #11
Collections

Collections - multiple properties to
assert
● Is null?
● Size
● Order
● Content

Collections
● Most of the time you want to assert on collection content

Collections
● Most of the time you want to assert on collection content
● Prefer exact content matching

Collections
● Most of the time you want to assert on collection content
● Prefer exact content matching
● Avoid incomplete assertions

Collections
● Most of the time you want to assert on collection content
● Prefer exact content matching
● Avoid incomplete assertions
● Do not sort just because it is easier to assert!

Collections
● Most of the time you want to assert on collection content
● Prefer exact content matching
● Avoid incomplete assertions
● Do not sort just because it is easier to assert!
● Multiple assertions are worse than single content
assertion

Collections
● Most of the time you want to assert on collection content
● Prefer exact content matching
● Avoid incomplete assertions
● Do not sort just because it is easier to assert!
● Multiple assertions are worse than single content
assertion
● Unless you want to say something important in your test!

Collections
● Most of the time you want to assert on collection content
● Prefer exact content matching
● Avoid incomplete assertions
● Do not sort just because it is easier to assert!
● Multiple assertions are worse than single content
assertion
● Unless you want to say something important in your test!
● Use matchers!

Thing #12
Access modifiers

Access modifiers
● Rule is simple - never access anything that
is not public in your tests

Access modifiers
● Rule is simple - never access anything that
is not public in your tests
● Private things are implementation details
which are not part of the public contract

Access modifiers
● Rule is simple - never access anything that
is not public in your tests
● Private things are implementation details
which are not part of the public contract
● Same applies for protected/package
modifiers. They must be there for production
code, but not available to your tests

Thing #13
Random values

Random values in tests
● Most of the time you do not want it

Random values in tests
● Most of the time you do not want it
● Unless you depend on randomness a lot (eg.
password generation*)
*Thanks to Aleksandar Tomovski for a good example

Random values in tests
● Most of the time you do not want it
● Unless you depend on randomness a lot
● Use property based testing (which is also
hard)

Random values in tests
● Most of the time you do not want it
● Unless you depend on randomness a lot
● Use property based testing (which is also
hard)
● Do not make dummy values random

What if we still need random cases?

Generate Multiple Test Cases
● Quality over quantity

Generate Multiple Test Cases
● Quality over quantity
● Think of boundary cases, that you may want
to detect with random test

Generate Multiple Test Cases
● Quality over quantity
● Think of boundary cases, that you may want
to detect with random test
● Use parameterized tests

Generate Multiple Test Cases
● Quality over quantity
● Think of boundary cases, that you may want
to detect with random test
● Use parameterized tests
● Random is hard to repeat

Generate Multiple Test Cases
● Quality over quantity
● Think of boundary cases, that you may want
to detect with random test
● Use parameterized tests
● Random is hard to repeat
● Flickering tests

Thing #14
How many assertions per test?

How many assertions per test?
● Unit test - one assertion per test. Must be
clear and readable
● Proper unit tests should fail for exactly one
reason
● End to end - best case one assertions per
test, but more allowed
● Consider custom matchers

Thing #15
Test Doubles

Test Doubles
The name comes from the notion of a Stunt
Double in movies

Why do we need test doubles?
● To test in an isolated environment by
replacing real collaborators with doubles
● To have fast tests
● To test interactions
● To change collaborators behaviour in test

Types of Test Doubles
● Dummy
● Fake
● Stub
● Spy
● Mock

Dummy
Dummy objects are passed around but never
actually used. Usually they are just used to fill
parameter lists

Fake
Fake objects actually have working
implementations, but usually take some
shortcut which makes them not suitable for
production (an in memory database is a good
example)

Stub
Stubs provide canned answers to calls made
during the test, usually not responding at all to
anything outside what's programmed in for the
test. Stubs may also record information about
calls, such as an email gateway stub that
remembers the messages it 'sent', or maybe
only how many messages it 'sent'

Spy
Stubs that verify behavior

Mock
Mocks are what we are talking about here:
objects pre-programmed with expectations
which form a specification of the calls they are
expected to receive

So what’s the difference
between Spy and Mock?

Thing #16
Naming

Where to keep your tests
● Recommendation: keep them in separate
package from production code

Class Naming
● End test classes with Test suffix (or one
required by your test framework)
● For long test classes:
○ split by feature
○ by input type

Method Naming
● Everybody should follow the same pattern
● testGeneratePassword vs
shouldGenerateValidPassword

Naming Test Doubles
● Intention revealing stubEngine, spyServer,
dummyUser, mockService

Thing #17
Comments

Comments in Test code
● Fundamentally good option to explain
complicated parts, but:
● better use good method naming
● custom matcher
● do less, so that intention is clear
● comments are not so bad in isolated well
named set up method (not the first thing to
be seen in test method)

Thanks!
Q & A