This document provides an overview of testing concepts and best practices. It discusses different types of bugs, the benefits of writing unit tests, test types like unit tests and integration tests, testability factors, isolation techniques using test doubles, common mistakes to avoid, and Microsoft Fakes for improving testability. The key points covered are: - The importance of writing tests to prevent bugs and allow safe refactoring. - Techniques like dependency injection, interfaces, and seams to make code more testable and allow isolation using stubs, fakes, mocks, and shims. - Common mistakes for test code like testing multiple features in one test, long test runs without maintenance, lack of negative testing, and unclear

1. Getting Started
With Testing
Topics:
Bugs
Tests
Unit Tests
Testability
Isolation
Common Mistakes

2. BUGS

3. What Bugs You?
Logical
Bugs
Wiring
Bugs
UI
Bugs

4. A Bug’s Life
1 X
500 X
DEV PRODUCTIONQADEV TEST LAB
The cost of a bug can increase 500 times from
development to the production release.
DESIGN
0.01
X
5 X
50 X

5. Ineffective Prevention
We should code fewer bugs
We need a tool that finds all bugs
We should hire more QA people
We should have (manual) test plans
We should refactor (without tests)
There is nothing to do about it
Oops…

6. Effective Prevention
 Good Design (complexity & coupling)
 High-Level technical documentation
 Code Reviews – Fagan, Over The Shoulder, Remote
 Pair Programming
 Use logs, error logs, e-mail notification, etc.
 Write Tests (especially Unit Tests)

7. TESTS

8. Test Types

9. The Test Pyramid
Black Box
White Box
White Box
END TO
END
INTEGRATION
TESTS
UNIT TESTS

10. UNIT TESTS

11. Benefits of writing Unit Tests
1. Bugs are found (much) earlier.
2. Safer changes/refactoring (regression).
3. Up-to-date technical documentation and
examples.
4. Better low level architecture.

12. I do not write tests because…
I do not know how.
It is not my job. The QA people are paid to do that.
It is not my job. Let the customer figure what’s wrong.
I work with untestable legacy code.
I only develop UI.
It makes it difficult to refactor the code.
Requirements are not clear.
I can test everything manually.
There is no time for that.

13. Anatomy of a Unit Test
Set up the needed
dependencies/para
Select a story (class
method + scenario)
Instantiate the class
Exercise/Stimulus
Run the method
Assert the expected
behavior
[TestClass]
public class CalculatorTest {
[TestMethod]
public void ShouldAddTwoPositiveIntegerNumbers() {
int addendum1 = 1;
int addendum2 = 2;
Calculator calc = new Calculator();
int sum = calc.Add(addendum1, addendum2);
Assert.AreEqual(sum, 3);
}
}

14. On-Field Mission:
Testing The Order Processor
Your Mission, should you choose to accept
it, is to unit test ValidateOrderTotal.
INPUT:
A customer id
OUTPUT:
True is the customer order for next delivery exists and the total
amount is > 0. False otherwise.
OrderProcessor processor = OrderProcessor.GetInstance();
bool isValidOrderTotal = processor
.ValidateOrderTotal("00000345-00000-000056");

15. Mission Impossible?
Mission log:
Attempt # Problem Solution
1 File Not Found Need configuration file
2 Invalid Database Set valid connection string
3 Invalid Ross Customer Id Set valid Ross Customer ID
4 Folder not found The export folder must exists and have
permissions
5 SMPT error Update SMPT configuration with valid server,
credential and set notification email
6 Order Processor Stuck Exception Delete marker file
7 Customer does not have order for next del.
date
Create order in DB for customer
8 What about the following delivery? …
… $#%k! …
… … …$#%k! $#%k!
[TestMethod]
public void CheckCustomerWithValidOrderForNextDeliveryDate() {
OrderProcessor processor = OrderProcessor.GetInstance();
bool IsValidOrderTotal = processor.ValidateOrderTotal("00000345-00000-000056");
Assert.IsTrue(IsValidOrderTotal, "...some error message...");
}

16. Bro! Seriously?!
There should be only one reason for a unit test to fail…
Do you have the configuration file?
What is the database connection
string?
Did you set up the export folder?
Did you set up a valid SMTP server?
Did you delete the marker file?
Will the test work next week?Did you set up the Ross Customer
ID correctly?
Does the customer key exist?
Does the customer have an order
for the next delivery?
Does the customer have an order
for the next delivery?
Did you set up a customer with a
valid order total?
Did you set up a customer with an
invalid order total?
Did you set up a customer with a
valid order total?
Did you set up the maximum
waiting time?

17. Corpus
Delicti
public class OrderProcessor {
private static OrderProcessor _instance = new OrderProcessor();
private bool _isMarkerFilePresent = false;
private int _rossCustomerId = 0;
private string _exportPath = string.Empty;
private OrderProcessor() {
_exportPath = ConfigurationSettings.AppSettings["OrderExportPath"];
_isMarkerFilePresent = IOHelper.CheckMarkerFile(this._exportPath);
_rossCustomerId = Convert.ToInt32(ConfigurationSettings.AppSettings["RossCustomerID"]);
int maxSecs = Convert.ToInt32(ConfigurationSettings.AppSettings["MaxSecsToWait"]);
int secsWaited = 0;
while (_isMarkerFilePresent) {
Thread.Sleep(1000);
secsWaited += 1000;
if (secsWaited > maxSecs) {
string email = DBService.GetRossCustomerEmail(_rossCustomerId);
EmailManager.SendEmail(email, "Order Processor is stuck! Check the marker file!");
throw new Exception("Order Processor is stuck! Check the marker file!");
}
}
IOUtility.CreateMarkerFile(_exportPath, "OrderProcessor.MRK");
}
public static OrderProcessor GetInstance() { return _instance; }
public void ExportOrder(string customerId) { … }
public bool ValidateOrderTotal(string customerId) {
decimal orderTotal = 0;
Order order = DBService.GetOrder(customerId, DateTime.Now);
if (order != null) {
foreach (OrderDetail orderDetail in order.Details) {
orderTotal += orderDetail.TotalAmount;
}
}
return orderTotal>0;
}

18. TESTABILITY

19. The Three Factors
Easy-to-write
meaningful
automated tests
Consistent
behavior across
multiple
deployments
Introspection

20. Testable vs. Untestable
Dependency Injection/Transparency
Interfaces/Factories/IOC
Law Of Demeter
Seams
Single Responsibility/Layered Architecture
Composition
Simple Constructors
Idempotence
Commutativity
Implicit Dependencies
New Operator
Service Locators
Static
Mixed Concerns
Deep Inheritance
Complex Constructors
Singletons, DateTime.Now/Random
Implicit Order (hints: Init, Pre, Post, Cleanup, Load)

21. A Better Point of View
Writing tests requires the code to be
testable.
Refactoring code for testability
requires time, skills and experience.
Before writing code, answer this question:
How would you like to test it?

22. Test First
Tests First = testability + small, well-isolated components
Focus on the API first and implementation later
Where did the database go?
Where is the marker file?
What happened to the configuration?
[TestMethod, TestCategory("Order Validation")]
public void OrderWithTotalAmountGreaterThanZeroShouldBeValid() {
Order order = CreateFakeOrder(totalAmount: 10.0);
IOrderValidation orderValidation = new OrderValidation(order);
bool isValidOrderTotal = orderValidation.IsValidTotal();
Assert.IsTrue(isValidOrderTotal,
"Total order amount > 0 should be valid");
}
[TestMethod, TestCategory("Order Validation")]
public void OrderWithTotalAmountZeroShouldBeInvalid() {
Order order = CreateFakeOrder(totalAmount: 0.0);
IOrderValidation orderValidation = new OrderValidation(order);
bool isValidOrderTotal = orderValidation.IsValidTotal();
Assert.IsFalse(isValidOrderTotal,
"Total order amount = 0 should be invalid");
}
[TestMethod, TestCategory("Order Validation")]
public void EmptyOrderShouldBeInvalid() {
Order order = Order.Empty;
IOrderValidation orderValidation = new OrderValidation(order);
bool isValidOrderTotal = orderValidation.IsValidTotal();
Assert.IsFalse(isValidOrderTotal,
"Empty Order should be invalid");
}
public class OrderValidation : IOrderValidation {
Order _order = Order.Empty;
public OrderValidation(Order order) {
_order = order;
}
public bool IsValidTotal() {
decimal orderTotal = 0M;
if (!_order.IsEmpty()) {
foreach (OrderDetail orderDetail in _order.Details) {
orderTotal += orderDetail.DetailAmount;
}
}
return orderTotal > 0;
}
}

23. ISOLATION

24. Controlling the
Test Environment
A good unit test is like a good scientific
experiment: it isolates as many variables as
possible (these are called control variables)
and then validates or rejects a specific
hypothesis about what happens when the one variable (the
independent variable) changes.
Test Code
System Under
Test (SUT)
Dep. 1
Dep. 2
Dep. NISOLATED TEST
SEAMS

25. System Under Test
SUT COLLABORATOR
PILOT PLANE

26. Test Doubles
Do you need a real plane?
We do not have a real plane available yet
It takes time and it is expensive to prepare a real
plane (checkup, gas, etc.)
Too dangerous, the plane may crash
If there is an issue it is hard to tell if the problem
was the pilot or the plane

27. Pick your double
Fake Mock
Dummy Stub/Spy

28. Dummy
Dummy objects are passed around but never
actually used. Usually they are just used to fill
parameter lists.
public class DummyPlane implements IPlane {
public DummyPlane() {
// Real simple; nothing to initialize!
}
public int getAltitude() {
throw new RuntimeException("This should never be called!");
}
public int getSpeed() {
throw new RuntimeException("This should never be called!");
}
}

29. Stub/Spy
Stubs provide canned answers to calls
made during the test, usually not
responding at all to anything outside what's programmed in for
the test. Spies can record some information based on how
they were called
public class PlaneStub implements IPlane {
public PlaneStub () {
// Real simple; nothing to initialize!
}
public int getAltitude () {
log.Write(“getAltitude was called”);
return 300;
}
public int getSpeed () {
throw new RuntimeException("This should never be called!");
}

30. }
Fake
Fake objects actually have working
implementations, but usually take some shortcut which makes
them not suitable for production (an In Memory Database is a
good example).
public class FakePlane implements IPlane {
…
public FakePlane (FlightStatus status) {
_flightStatus = status;
}
public int getAltitude () {
return CalculateAltitude(_flightStatus);
}
public int getSpeed () {
return CalculateSpeed(_flightStatus);
}
}

31. Mock
Mocks are pre-programmed with expectations
which form a specification of the calls they are
expected to receive. They can throw an exception if they
receive a call they don't expect and are checked during
verification to ensure they got all the calls they were
expecting.
planeControl = MockControl.createControl(Plane.class);
planeMock = (IPlane) planeControl.getMock();
public void testPilotCheckAltitude() {
Pilot pilot = new Pilot();
planeMock.getAltitude();
planeMock.setReturnValue(300);
planeMock.replay();
pilot.Flight((IPlane) planeMock);
planeControl.verify();
}

32. MICROSOFT FAKES

33. What is it?
Microsoft Fakes is a code isolation framework that can help you isolate code for
testing by replacing other parts of the application with stubs or shims. It allows
you to test parts of your solution even if other parts of your app haven’t been
implemented or aren’t working yet.

34. Stubs
[TestMethod]
public void
StubAccountOpenPostsInitialBalanceCreditTransaction()
{
// Arrange:
int callCount = 0;
StubITransactionManager stubTM = new StubITransactionManager
{
PostTransactionDecimalTransactionType = (amount, type) =>
{
if (amount == 10m && type == TransactionType.Credit)
{
callCount++;
}
}
};
Account cut = new Account(stubTM);
// Act
cut.Open(10m);
// Assert
Assert.AreEqual<int>(1, callCount);
}

35. Shims (Monkey Patching)
Shims are a feature of Microsoft Fakes that allows creating
unit tests for code that would otherwise not be testable in
isolation. In contrast to Stubs, Shims do not require the
code to be tested be designed in a specific way.
[TestMethod]
public void DateTimeTes()
{
using (ShimsContext.Create())
{
// Arrange:
System.Fakes.ShimDateTime.NowGet = () => { return new
DateTime(2016, 2, 29); };
// Act
int result = MyCode.DoSomethingSpecialOnALeapYear();
// Assert
Assert.AreEqual(100, result);
}
}

36. COMMON MISTAKES

37. All-In-One
This is by far the most common mistake among beginners.
It comes in two flavors:
1. Unrelated features tested in the same test class/method
2. Single feature tested across multiple layers
Consequences: when the test fails, it is hard to point out at
what has failed
Long Parking
Tests are written correctly but they are executed rarely
and not maintained
Consequences: obsolete tests do not help preventing
bugs. They could mislead developers who read them to
try to understand the program

38. Where is the negative?
Other common mistake, tests are checking only what
the application should do (positive cases) and not
what it should not do (negative cases). Edge cases
and expected exception are also typically ignored by
beginners.
Consequences: poor coverage due to untested scenarios
Bad Names
Tests methods have names that are too
generic or meaningless. E.g. “reportTest”
or “authenticationTest”
Consequences: when the test fails, it is
not immediately obvious what has failed.
They could mislead developers who read
them to try to understand the program

39. Do not succumb to the Dark Side
Write Unit Tests!
Further Reading
Guide: Writing Testable Code
Exploring The Continuum of Test Doubles
Better Unit Testing with Microsoft Fakes
Videos
Introduction to Unit Testing in .NET
http://dotnet.dzone.com/articles/introduction-unit-testing-net
The Clean Code Talks -- Unit Testing
http://www.youtube.com/watch?v=wEhu57pih5w
GTAC 2010: Lessons Learned from Testability Failures
http://www.youtube.com/watch?v=4CFj5thxYQA