Ever tried doing Test First Test Driven Development? Ever failed? TDD is not easy to get right. Here's some practical advice on doing BDD and TDD correctly. This presentation attempts to explain to you why, what, and how you should test, tell you about the FIRST principles of tests, the connections of unit testing and the SOLID principles, writing testable code, test doubles, the AAA of unit testing, and some practical ideas about structuring tests.

1. TDD/BDD & Unit Testing
Best Practices

2. Attila Bertók
C# technical lead
bertok.atti@gmail.com
https://www.linkedin.com/in/bertokattila/

3. Contents at a Glance
 Why Test?
 What to Test?
 FIRST Tests, the 5+3 Requirements of Tests
 Unit Testing and Solid
 Writing Testable Code
 Test Doubles
 The AAAs of Unit Testing
 How to Test
 Structuring Tests

4. Why Test?
Just write code that works, duh!

5. “
”
It is unit tests that keep our code flexible,
maintainable, and reusable. The reason is simple. If you
have tests, you do not fear making changes to the code!
Without tests every change is a possible bug. No matter
how flexible your architecture is, no matter how nicely
partitioned your design, without tests you will be
reluctant to make changes because of the fear that you
will introduce undetected bugs.
[Clean Code – Robert C. Martin]
 Tests are there for the developer to enable refactoring.
 Tests are there for the developer and the PO to provide a living specification.
 Tests are there for the developer to provide a living documentation.
 Tests are there for the business to provide a fast feedback loop for catching bugs
early.

6. The Cost of Fixing Bugs in Different
Phases of the SDLC
Design Coding Testing Post-Shipping
1 unit 6 units 15 units 60-100 units
 Factors:
 The length of the feedback loop
 Time it takes for the information to propagate back to the developer
 The context switch cost for the developer
 The design modification consequences
 Having to modify the design after it is already implemented is costly
 Modifying existing code might just be exchanging known bugs for unknown bugs
 Not in a well-tested application, though

7. Implications
 Design well
 Have as short a feedback loop between design and development as possible
(do you even agile, bro?)
 Move as much of your testing to development-time as possible
 Automate tests as much as possible

8. What to Test
Test Everything

9. Test I/O, Not Internals, part 1
 Suppose your task is to implement sorting on a list of integers
 You remember bubble sort, because, well, that’s easy. You implement it,
together with 100% test coverage, in line with your DoD.
 Your test case looks like this:
 Given the input of 8, 2, 7, 3, 5
When I run the sorting
Then after the first step, my list will be 2, 8, 7, 3, 5
And after the second step, my list will be 2, 7, 8, 3, 5
And after the third step, my list will be 2, 7, 3, 8, 5
And after the fourth step, my list will be 2, 7, 3, 5, 8
And after the fifth step, my list will be 2, 3, 7, 5, 8
And after the sixth step, my list will be 2, 3, 5, 7, 8

10. Test I/O, Not Internals, part 2
 The next day, your PO tells you, that your algorithm is not good enough, when
used on the 300 million records of the production database, results take ages
to compute.
 You dust your Algorithms book off, look up QuickSerach, and implement it.
 However, you have to write a whole new set of tests, as after one step of
QuickSort, the list will be 5, 2, 7, 3, 8, not 2, 8, 7, 3, 5.
 Should you have only tested that
 Given the input of 8, 2, 7, 3, 5
When I run the sorting
Then the sorted list is 2, 3, 5, 7, 8
then the test would have kept working, even after changing the underlying
algorithm.

11. Anatomy of Tests
 A test should consist of some input, some action(s) to take, and some output.
 Internal steps and interim states should not be validated.

12. Coverage
 Code coverage tools measure how thoroughly tests exercise
“programs”/“code”.
 Statement coverage: record if the line of code was executed
 Multiple condition coverage: record if each logical condition in the code was
evaluated for both the true and the false branch
 Is there a thing such as “high enough” coverage?

13. “
”
How much of the code should be tested with these automated unit
tests? Do I really need to answer that question? All of it! All. Of. It.
Am I suggesting 100% test coverage? No, I’m not suggesting it. I’m
demanding it. Every single line of code that you write should be
tested. Period.
Isn’t that unrealistic? Of course not. You only write code because you
expect it to get executed. If you expect it to get executed, you
ought to know that it works. The only way to know this is to test it.
[Clean Coder – Robert C. Martin]
Anyways, if you work in test first style TDD, your coverage will automatically be
very close to 100%.

14. The Test Pyramid

15. The Test Pyramid, Part 2
 System/E2E tests: load testing, usability
testing, performance testing,
compatibility testing, stress testing,
security testing, regression testing,
installation testing, etc. Partially
automated, partially manual, each type
covers a specific aspect of the system.
 Automated UI tests
 Integration tests: difficult to write, might
take long to run. Covers the “plumbing”.
 Component/Acceptance tests: easy to
write and run, easy to obtain high
coverage, should cover any part of the
code that provides business value.
 Unit tests: easy and fast to write, run,
should cover most of the code. (All of the
code, if using test first TDD.)
Manual
Exploratory
Tests
Automated UI
Tests
System/E2E Tests
Integration Tests
Component/Acceptance Tests
Unit Tests

16. Who Creates What?
Tester skill
required
Manual
Exploratory
Tests
Automated UI
Tests
System/E2E Tests
Integration Tests
Component/Acceptance Tests
Unit Tests
Developer
skill
required

17. Visibility and Unit Testing
 Unit tests should test public methods (or, being more strict, public
interfaces might be an even better choice).
 Don’t make a method public just in order to make is testable.
 What about internal classes/methods?
 Can work, use the InternalsVisibleTo attribute!
 Should we have internal classes/methods anyways?
 Think of APIs or adding your dll assembly as a reference to another Visual Studio
project, and seeing what’s visible to external developers.
 (So yes, we should. Most likely that’s what you should be using instead of public.)

18. Difference between Unit Tests and
Acceptance Tests
 Consider your tests as living specification of your application.
 Acceptance tests are the business-level specification. Human-readable, with
test cases that are meaningful for the business. Used for feature validation.
 Unit tests are the developer-level specification, that check how
functions/methods in the code work. The business does not really need to
understand the technical details, but other developers do. Used for
functionality verification.

19. FIRST Things First
Thumb Rules of Unit Tests

20. FIRST
 Fast
 Independent
 Repeatable
 Self-Validating
 Timely
 +1 T: Thorough and Reliable.
 +2 Es: Easy to Read, Easy to Write

21. Fast
 How long are you willing to wait
for your unit tests to run?
 Remember the TDD cycle?
 Let me remind you…

22. Fast

23. Fast
 You…
 … create your test class
 … reference your subject, new it up
(mock dependencies)
 … create your test method
 … make your test method fail
(Assert.False)
 … call the method you want to test
(and generate it with the body
throwing a
NotImplementedException)
 … run all your unit tests
 … create the dumbest
implementation possible that makes
your unit test pass (return the value
that your test expects)
 … run all your unit tests
 … create another test method with
different input
 … run all your unit tests
 … implement your business logic
 … run all your unit tests
 … create test methods for edge cases
 … run all your unit tests
 … etc.

24. Fast
 You probably end up having thousands of tests.
 You run all of your tests about 3-5 times for each method you write.
 How long are you willing to wait for them to run?
 Developers should not hesitate to run the tests because they are slow.
 Meaning that tests should run (including setup, the actual run, and teardown)
in a matter of milliseconds each, so the sum running time of all tests should
not be any longer than a couple of seconds.

25. Independent
 Test running order should not interfere with the test result.
 Some test runners enforce this by running your tests in a random order.
 Running the tests one-by-one or in a bundle should not interfere with the
result.

26. Repeatable
 Tests should be deterministic: every run of the same test code on the same
subject code should come up with the same result.
 Do not depend on external data or environment the test is running in. Setup /
mock whatever is use in the test method.
 Do not depend on date/time either!
 Each test should arrange or setup its own data.
 If some tests need common data, use some sort of data context for them. (Lots of
test frameworks provide those.)
 Do not use random input. Use sensible input, but with such values that it is
visible from the first glance that the input is not production data.
 You need a name? Use “TestName”.
 You need an 8-digit number? Use “12345678”.

27. Repeatable
 Do preprocessing in the Setup
 Prepare environment if needed (setup mocks, flags, settings, sample data, etc.)
 Do post-processing in the TearDown
 Cleanup to the original state after the test has run
 Remove created files, database entries, etc.

28. Self-Validating
 The test should provide a binary output: Passed/Failed, Green/Red.
 (Or ignored)
 No manual inspection (checking log files, validating anything by hand) should
be required to validate the test results.

29. Timely
 Tests should be written together with the code.
 Covering existing code with unit tests later is difficult. (Anyone knows PEX?)
 And it is rare that you have the time for doing that, anyways.
 Testability influences code design. You don’t just write tests to have your code
tested, you write tests to aid you in designing SOLID code.
 Unit testing provides a genuine need for implementing and following the SOLID principles.
 To put it another way: The harder it is to unit test a piece of code, the greater the
chances that SOLID principles are being violated.
 We’ll get back to this in just a moment…

30. +1T: Thorough and Reliable
 Don’t just aim for a high coverage (100% should be the baseline anyways in test
first), aim for testing scenarios.
 It is possible that there are cases you did not think about when writing your unit test.
Because of this, your might be 100%, but as soon as you create your new scenario for the
test case you did not think about previously, you realize the need for additional
production code, meaning that there was a deficiency in the code under test, in spite of
the high coverage.
 If there is a bug in the system under test, at least one unit test should fail.
 Creating and running failing unit tests in the TDD cycle serves the purpose of validating
that the test can actually fail. The fact that the test can fail should be validated,
otherwise the test can be a false positive, that passes no matter the bugs in the code.
 Tests should be a living documentation of your code.
 Whatever your code is doing should be understandable from your test suite.
 To achieve this, you also need high coverage.

31. +1E: Easy to Read
 Unit tests are living documentation.
 This means that:
 the tests should
 be well-named
 have a clear intent
 the scenario should
 be easy to understand
 tell a story of a use-case or a functional aspect
 in case of a failure
 the cause of the failure should be obvious
 the steps to take to fix the failure should be obvious

32. +1E: Easy to Write
 As the coverage should be around 100%, there will be lots of tests. Writing
(modifying, maintaining) tests should not be difficult.
 SOLID code helps a lot here.
 Some libraries also tend to help a lot. Use a mocking library, use an assertion
helper.
 Some libraries also have extensions / subparts to help in testing them
(mock/dummy implementations, etc.). Use those as well.

33. Unit Testing and SOLID
How Does Unit Testing Help You Write SOLID Code?

34. Single Responsibility
 If your class violates the SRP, you will have massive test classes.
 Class Size smell for the test class. You will generally end up 2-8 times the
number of test methods than the methods being tested.
 The happy path and at least one unhappy path, can be way more based on the
parameters Say, double:
 0.0, 15.0, -15.0,
 double.MaxValue, double.MinValue,
 double.NaN, double.NegativeInfinity, double.PositiveInfinity.
 Some of your tests use some dependencies, other tests don’t use them.
 This is abusing dependencies, a violation of the ISP as well.

35. Open/Closed
 Injecting code that allows testing opens production code for extensibility.
 Showing a MessageBox is difficult to test. However, injecting an Action<string> (or
an INotificationProvider) instead of calling MessageBox.Show(message) opens the
code to extension: notification by email, writing to a database (or using a
messagebox) becomes possible – the same way, a mock implementation that
facilitates testing also becomes possible.
 And the class loses a responsibility as well.
 Unit tests indirectly close the class for modifications: modifications will break
some of the tests.

36. Liskov Substitution
 This is the reason our mocks work: the code under test cannot tell the
difference.
 Code under test should not know it is running in “test mode”.
 Unit tests should work on derived classes without any modifications.
Otherwise inheritance might not be the best option.

37. Interface Segregation
 Mocking is generally done through an interface. If you find yourself needing to
mock a large interface or a complex interaction of methods you should think
about the design of those dependencies and about splitting those interfaces
or interactions into smaller chunks, so that you can mock only the chunk that
concerns your code under test right now.
 A badly designed interface will make mocking difficult or will need several
different mocks depending on how it is being used.

38. Dependency Inversion
 Some dependencies (database, web service, logger, etc.) may make your unit
test behave like an integration test. This is a sure sign that these
dependencies need to be mocked – thus, injected.
 As the mock will generally not be the class itself, but a mock class that
implements the same interface, this also enforces to depend on abstractions
instead of implementations.

39. Writing Testable Code
Or the plague that is a Singleton

40. Not All Production Code Are Born Equal
 Unit testing works best on… units.
 Units can be separated effectively from their environment
 Remember Dependency Inversion and Interface Segregation, aim for Loose Coupling
 Dependencies should be made explicit and the new() operator should be avoided
 Mocks should be injectable instead of these dependencies
 Non-deterministic factors should be mockable (no hidden DateTime.Now or
Environment.MachineName in the middle of the method) with a deterministic mock
 Side-effects should be avoided (frequently by passing method delegates in as
dependencies)
 Static application state (static properties and static fields) should be avoided
 Singletons should be avoided (singletons are hidden application states)
 The Command and Query Segregation principle should be applied

41. Isolating Tests
 Dependencies of the code under test should
 be deterministic
 not break the tests
 Issue: production code (or the user, for that matter) is rarely deterministic
 Solution: test should not depend on the environment
 And this environment includes the dependencies of the class under test
 Test Doubles allow you to mimic the behavior of classes and interfaces,
letting the code under test interact with them as if they were real. This
isolates the code you’re testing, ensuring that it works on its own and that
no other code will make the tests fail.

42. Test Doubles
Stunt Doubles for Production Code

43. Test Doubles
 Dummy:
 Simple placeholder. Can be as simple as a null, a constant, or a method
implementation that throws NotImplementedException.
 Fake:
 Working implementation that is a simplification of the production version. E.g.
faking a data repository with a non-persistent in-memory collection.
 Stub:
 An implementation that holds pre-defined data and returns that during the test.
The input is configured from the test set-up or the test arrange.
 Mock:
 An implementation that records calls it receives. Can be used to validate that calls
to external dependencies happened, without caring about what the call returned.

44. Dummy
Placeholder that is not used during test execution. Can be as simple as a null, a
constant, or a method implementation that throws NotImplementedException.
public class Warehouse : IWarehouse {
public Warehouse(string location) { // … }
public void PlaceOrder(string productId, int amount) {
// …
}
}
public void OrderIsFulfilled_IfThereAreEnoughItemsInStock() {
IWarehouse w = new Warehouse(null);
w.SetStock(…); // …
}

45. Fake
Working implementation that is a simplification of the production version.
public class FakeAccountRepository : IAccountRepository {
IDictionary<,> accounts = new Dictionary<IUser, IAccount> {
{ new User(“john@mail.com”), new UserAccount() },
{ new User(“bob@xmail.com”), new UserAccount() }
}
public bool IsPasswordValid(User user, string pw) {
return pw == accounts[user].GetPasswordHash();
}
}

46. Stub
An implementation that holds pre-defined data.
public class FakeDie : IDie {
public int Roll() {
// chosen by fair dice roll.
// guaranteed to be random.
return 4;
}
}

47. Mock
An implementation that records calls it receives. Used for testing behavior
instead of state.
warehouseMock
.Setup(w => w.PlaceOrder(It.IsAny<string>(), It.IsAny<int>()))
.Verifiable();
warehouseMock.PlaceOrder();
// Placing an order should check the stock
wareHouseMock.Verify(warehouseMock => warehouseMock.CheckStock(),
Times.Once);

48. AAA
BYJCHEMA
(Because You Just Can’t Have Enough Mnemonic Acronyms)

49. The AAA
 Arrange: Setup the data, mocks, environment. Data setup is part of the test.
 Act: Invoke the method that is being tested.
 Assert: Check the results.
 Fun fact: Mathematically this is a Finite State Machine:
 Arrange is the Starting State
 Act is a State Transition
 Assert is the expected Finishing State

50. The Extended AAA
AAA
Arrange
Act
Assert
Test
SetUp
Test Method
TearDown
TestFixture
TestFixtureSetUp
Multiple Tests
TestFixtureTearDown
A
A
A
T
e
s
t

51. AAA Best Practices
 Arrange
 Don’t have an extensive Arrange block. Having too many things to set up is most
likely a sign of violating the SRP.
 Do not Assert in your Arrange phase.
 Invalid input data should probably throw an exception.
 Consider using Code Contracts.
 Assert
 Single Concept per Test
 Prefer one single assert, or a tightly knit group of asserts, per test method.
 Very similar to the single point of return principle.

52. How to Test?
Unit Test Project Best Practices

53. Coding Standards for Unit Tests
 Apply the same quality standards (coding standards, reviews, pull requests, CI
builds) that you apply for your production code.
 Tests must change as production code changes. The more dirty the tests are,
the more difficult it will be to maintain them.
 Old tests will start to fail, and will be difficult to fix.
 More time will be spent on fixing failing tests than writing production code.
 Tests will be viewed as a liability, and developers will be tempted not to run the
tests.
 Test code is just as important as production code.

54. Taking Unit Tests Seriously
 All tests should be run on the developer machine before checking in.
 All tests should be run after building, both on the developer machine and on
the CI server.
 In a CI environment this means that tests should be run on all check-ins.
 On all the branches, optimally.
 Failing tests should break the build.
 Failing a test should be considered an equal offense to committing non-
complying code.
 When using gated check-ins or pull requests, failing tests should prevent the
check-in.

55. Ignoring Tests
 Avoid ignoring a test that you broke. Refactor, cleanup, fix, don’t ignore.
 Don’t ignore if the code needs to be fixed, not the test. You are hiding a bug with
the ignore attribute.
 Avoid ignoring if the test needs to be fixed. You most likely will never have the
chance to return to fix the test for the same time concerns you might chose to
ignore it when you broke it.
 Should you need to ignore, avoid Ignore attributes without explanation.

56. How to Test?
Test method creation best practices

57. Specification by Example
 Specification is difficult to code
 Examples are easy to code
Lets see an… example.
Consider the following user story:
As a Math idiot
I want to be told the sum of two integer numbers
In order to not make silly mistakes

58. Formal Specification vs. Specification by
Example
Acceptance criteria:
 I can enter two numbers
 The result is the sum of the two
numbers
 The result is the same irrespective
of the order I enter the numbers to
sum
 Adding zero to a number does not
change its value
Acceptance criteria:
Examples:
|value1|value2|
| 0| 1|
| 1| 2|
| 2| 1|
| -25| 5|
When I add <value1> to <value2>
And I add <value2> to <value1>
Then <result1> equals <result2>
When I add <value1> to 0
Then my result is <value1>

59. Property-Based Testing
 Choosing examples wisely is key
 Edge cases are often a good choice
 Apart from edge cases, the happy path values can be difficult to determine
 And the developer is biased to create test cases that pass
 However, if you can describe invariant properties of your method, you can use
these properties to create a good set of happy-path tests

60. Property-Based Testing, part 2
 Some examples of properties:
 Adding an item to the shopping cart increases the count of items in the shopping
cart
 Adding an integer value to zero results in the input integer value
 Multiplying an integer value by zero results in a zero value
 Sorting a list of integers results in the same amount of integers in a list
 Test for these properties. Either use a set of values that are easy to verify or
use (lots of) random values*
*: Didn’t we explicitly say that we should not test with random values? Well, yes, property-based testing might be an
exception to this rule – but it is not necessarily one. You can generate your input-values manually as well.

61. Fluent Assertions
 Which one is correct?
 Assert.AreEqual(actual, expected)
 Assert.AreEqual(expected, actual)
 Using multiple asserts for asserting one concept looks contrary to the single
assert per test principle
 Enter Fluent Assertions
 Compare the following assertions pairs and decide which ones are easier to
read

62. Readable Assertions
var subject = new Person();
MSTest Assert
Assert.IsTrue(subject.isOver18);
Assert.IsFalse(subject.isOver18);
Assert
.AreEqual(subject.MaidenName,
subject.CurrentName);
Assert
.AreNotEqual(subject.MaidenName,
subject.CurrentName);
Assert
.IsInstanceOfType(subject,
typeof(Person));
CollectionAssert
.DoesNotCointain(employees, subject);
FluentAssertions
subject.isOver18.Should().BeTrue();
subject.isOver18.Should().BeFalse();
subject.MaidenName
.Should()
.Be(subject.CurrentName);
subject.MaidenName
.Should()
.NotBe(subject.CurrentName);
subject.Should().BeOfType<Person>();
employees.Should()
.NotContain(subject);

63. Readable Assertions
var subject = new Person();
MSTest Assert
[ExpectedException(typeof(TooYoungToDrinkException))]
public void CannotDrink() {
subject.Drink();
}
FluentAssertions
public void CannotDrink() {
Action drink = () => subject.Drink();
drink.Should().Throw<TooYoungToDrinkException>();
}

64. Structuring Unit Tests
What do I call you?

65. Organizing Unit Tests, Part 1
 Where to place unit test projects?
 Generally speaking, they should be rather close to the project being tested.
 Certainly in the same solution.
 But test assemblies should not be deployed with the production code!
 Generally you will want to use the “ProjectName.UnitTests” folder and
namespace convention.
 There are religions that promote using the .UnitTests for the folder, but not for the
namespace. Experiment, whether you like it or not. (Downside is that Intellisense
will display both when writing code in the test class.)

66. Organizing Unit Tests, Part 2
 Mirror the folder/file structure (and of course namespace structure) of the
production code as closely as possible.
 Place test-specific classes (helpers, custom assertions, etc.) into a separate
folder (TestUtilities).
 Consider moving these to a shared dll that all your test projects can reference to
avoid code duplication.

67. Organizing Unit Tests, Part 3
 For each class under test create a test fixture class, name it
“[ProductionClassName]Tests”. (e.g. PersonTests)
 You will most likely want to make this class partial for size-related reasons.
 This will contain your “subject” or “unitUnderTest”, that is, the instance of the
class you are testing.
 For each method under test create a nested test fixture class that derives
from the containing class ([ProductionClassName]Tests). Name these after
the method that they test. (e.g. IsAllowedToDrink)

68. Organizing Unit Tests, Part 4
 Create test methods inside the [MethodName] classes.
 Naming conventions: IsAdult.[…]
 StateUnderTest_ExpectedBehavior:
IfAgeIsLessThan18_IsFalse
 ExpectedBehavior_StateUnderTest:
IsFalse_ForAgeLessThan18
 Should_ExpectedBehavior_When_StateUnderTest:
Should_BeFalse_When_AgeIsLessThan18
 When_StateUnderTest_Expect_ExpectedBehavior:
When_AgeIsLessThan18_Expect_ToBeFalse
 Use a “Cannot”/”DoesNot” syntax when expecting exceptions
 CannotAccept_NullArguments / DoesNotAccept_NullArguments

69. Summary
 Tests are not just validation and verification
 They are specification, documentation and design aids.
 They save you a lot of engineering effort and engineering hours (thus money)
 SOLID code is easy to test, easy to test code is most likely clean
 Loose coupling, CQS, DI are keys in testability
 Tests should be FIRST+TR+E2R+E2W
 Tests are input, action, and output validation
 AAA is a Finite State Machine
 Specification by Example conveys acceptance criteria clearly
 Tests are First Class Citizens in respect of standards and maintenance
 Libraries can make testing easy (SpecFlow, Fluent Assertions, FsCheck)

70. Thank You & Q&A