The secret
Unit Testing tools
no one has ever told you about
Dror Helper | http://helpercode.com | @dhelper
Consultant & software architect
Developing software since 2002
Clean Coder & Test Driven Developer
Pluralsight author
B: http://helpercode.com
T: @dhelper
About.ME
Background: unit testing tools
[Test]
public void AddItem_AddTwoValidProductsToCart_TotalEqualsItemsPrice()
{
var fakeRepository = A.Fake<IProductRepository>();
Product product1 = new Product(1, "product-1", 100);
Product product2 = new Product(2, "product-2", 200);
A.CallTo(() => fakeRepository.GetProductById(1)).Returns(product1));
A.CallTo(() => fakeRepository.GetProductById(2)).Returns(product2));
var shoppingCart = new ShoppingCart(fakeRepository);
shoppingCart.AddItem(1);
shoppingCart.AddItem(2);
Assert.That(shoppingCart.Total, Is.EqualTo(300));
}
(Most)
Unit Testing
Frameworks
[Test]
public void MyTest()
{
}
A good Unit Test should be:
Easy
to write
Simple
to read
Trivial
to maintain
What about AAA?
[Test]
public async void GetUserFromUrl() {
var clientFake = A.Fake<IJsonClient>();
A.CallTo(() => clientFake.HttpGetUncompressedAsync(A<string>.Ignored))
.Returns(Task.FromResult(JsonResult));
var userRepository = new UserRepository(clientFake);
var user = await userRepository.GetUser(11361);
var expected = new User {
Id=11361, DisplayName = "Dror Helper", ImageUrl=DefaultAvatar, Reputation=13904
};
Assert.That(user, Is.EqualTo(expected));
}
Arrange
 Act
 Assert
Test setup (Arrange)
Arrange issues
[TestMethod]
public async Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() {
var user1 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10};
var user2 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10};
var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 });
var viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository));
await viewModel.LoadUser();
await viewModel.LoadUser();
var result = await InvokeAsync(() => ((SolidColorBrush)viewModel.ReputationTrend).Color);
Assert.AreEqual(Colors.White, result);
}
[TestInitialize]
public async Task InitializeViewModel() {
var user1 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10};
var user2 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10};
var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 });
_viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository));
}
[TestMethod]
public async Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() {
await _viewModel.LoadUser();
await _viewModel.LoadUser();
var result = await InvokeAsync(() => ((SolidColorBrush)_viewModel.ReputationTrend).Color);
Assert.AreEqual(Colors.White, result);
}
Using Setup/TearDown
[TestInitialize]
public async Task InitializeViewModel() {
var user1 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10};
var user2 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10};
var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 });
_viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository));
}
[TestMethod]
public async Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() {
await _viewModel.LoadUser();
await _viewModel.LoadUser();
var result = await InvokeAsync(() => ((SolidColorBrush)_viewModel.ReputationTrend).Color);
Assert.AreEqual(Colors.White, result);
}
private UserDetailsViewModel _viewModel;
Abusing Setup
methods
Extract to methods
[TestMethod]
public async Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() {
var user1 = CreateUser(10);
var user2 = CreateUser(10);
var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 });
var viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository));
await viewModel.LoadUser();
await viewModel.LoadUser();
var result = await InvokeAsync(() => ((SolidColorBrush)viewModel.ReputationTrend).Color);
Assert.AreEqual(Colors.White, result);
}
The problem with Factories
private User CreateUser(int reputation) {
return new User
{
ImageUrl = "http://dummy.jpg",
Reputation = reputation
};
}
private User CreateUser(int reputation, string imageUrl) {
return new User
{
ImageUrl = imageUrl,
Reputation = reputation
};
}
Solution: Builder Pattern
class UserBuilder {
private int _id, _reputation;
private string _name, _imageUrl;
public UserBuilder() {
_id = 1;
_name = "dummy";
_imageUrl = "http://dummy.jpg";
}
User Build() {
return new User {
Id = _id, Name = _name, ImageUrl = _imageUrl, Reputation = _reputation
};
}
}
Builder Pattern (cont)
class UserBuilder {
...
public UserBuilder WithName(string name) {
_name = name
return this;
}
public UserBuilder WithReputation(int reputation) {
_reputation = reputation
return this;
}
...
}
var user1 = new UserBuilder()
.WithReputation(10)
.Build();
Tool: AutoMocking Containers
Test Container SUT
http://blog.ploeh.dk/2013/03/11/auto-mocking-container/
New()
Configure
CreateCreate SUT
Act
Verification (Assert)
Can you spot the problem?
[TestMethod]
public void PerformSomeActionReturns42()
{
var myClass = ...
bool initOk = myClass.Initialize();
var result = myClass.PerformSomeAction();
Assert.IsTrue(initOk);
Assert.AreEqual(42, result);
}
How about now?
[TestMethod]
public void TestPasswordComplexity()
{
var result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "1!").Result;
Assert.IsFalse(result.Succeeded);
result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "123456789").Result;
Assert.IsFalse(result.Succeeded);
result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "123456789!").Result;
Assert.IsFalse(result.Succeeded);
result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "abcdefghijk").Result;
Assert.IsFalse(result.Succeeded);
result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "abcdefghijK1!").Result;
Assert.IsTrue(result.Succeeded);
}
http://stackoverflow.com/q/26400537/11361
How many Asserts per test?
One Assert per test
2 Asserts == 2 Tests
Really???
Multiple Asserts can make sense
[TestMethod]
public void CompareTwoAsserts()
{
var actual = GetNextMessage();
Assert.AreEqual(1, actual.Id);
Assert.AreEqual("str-1", actual.Content);
}
HOW UNIT TESTING FRAMEWORKS HURT YOUR UNIT TESTING EFFORTS
WHY TESTS THROW EXCEPTIONS ON FAILURE?
It’s trivial, and best practice
Helps Decouple test runner from test
But not necessary…
public class AssertAll
{
public static void Execute(params Action[] assertionsToRun)
{
var errorMessages = new List<exception>();
foreach (var action in assertionsToRun)
{
try
{
action.Invoke();
}
catch (Exception exc)
{
errorMessages.Add(exc);
}
}
if(errorMessages.Any())
{
string errorMessage = string.Join("n", errorMessages);
Assert.Fail($"The following conditions failed:n{errorMessage}"));
}
}
}
https://helpercode.com/2011/02/16/multiple-asserts-done-right/
Using Assert.All
[TestMethod]
public void CompareTwoAsserts()
{
var actual = CreateMessage();
AssertAll.Execute(
() => Assert.AreEqual(1, actual.Id),
() => Assert.AreEqual("str-1", actual.Content);
}
Some frameworks are catching up!
https://github.com/nunit/docs/wiki/Multiple-Asserts
Assert.AreEqual(5, 2 + 2); Assert.AreEqual(2 + 2, 5)
Assert.AreEqual(5, 2 + 2); Assert.IsTrue(2 + 2 == 5)
How to choose the right Assert?
• IsTrue vs. AreEqual
• Parameter ordering confusion
• StringAssert/CollectionAssert
It’s all about proper error messages
AssertHelper
[Test]
public void CompareTest()
{
var myClass = new MyClass();
Expect.That(() => myClass.ReturnFive() == 10);
}
[Test]
public void CompareTest()
{
var c1 = new[] { 1, 2, 3, 4, 5 }
Expect.That(() => c1.Contains(42);
}
https://github.com/dhelper/AssertHelper
Assertion Libraries: FluentAssertions
[Fact]
public void CompareTwoObjects()
{
var customer1 = new Customer("cust-1", "John Doe");
var customer2 = new Customer("cust-2", "John Doe");
customer1.ShouldBeEquivalentTo(customer2,
o => o.Excluding(customer => customer.Id));
}
http://www.fluentassertions.com/
Test organization
Test structure issues
• What to call the test?
• AAA is not mandatory
• What should I test?
• How to avoid unreadable, complicated tests?
- Unit testing frameworks do not provide structure
The BDD approach
Step
Definitions
BDD Example: SpecFlow
http://www.specflow.org/
[TestClass]
public class CardHasBeenDisabled {
private Card _card;
private Atm _subject;
void GivenTheCardIsDisabled() {
_card = new Card(false, 100);
_subject = new Atm(100);
}
void WhenTheAccountHolderRequestsMoney() {
_subject.RequestMoney(_card, 20);
}
void ThenTheAtmShouldRetainTheCard() {
Assert.IsTrue(_subject.CardIsRetained);
}
void AndTheAtmShouldSayTheCardHasBeenRetained() {
Assert.AreEqual(DisplayMessage.CardIsRetained, _subject.Message);
}
[TestMethod]
public void Execute()
{
this.BDDfy();
}
}
Tool: BDDfy
Test execution
Continuous Testing Tools
DotCover
Typemock
Runner
nCrunch
Live Unit Tests
The right tools will help you write good tests
Arrange
Builder
Pattern
AutoMocking
Containers
Assert
Multiple
Asserts
3rd party
Assertion
libraries
Test
Structure
“Classic” BDD
In-Code BDD
Continuous
Testing
Typemock Runner
DotCover
nCrunch
Live Unit Testing
Corvlet (.NET Core)
Thank you
@dhelper | http://helpercode.com

The secret unit testing tools no one ever told you about

  • 1.
    The secret Unit Testingtools no one has ever told you about Dror Helper | http://helpercode.com | @dhelper
  • 2.
    Consultant & softwarearchitect Developing software since 2002 Clean Coder & Test Driven Developer Pluralsight author B: http://helpercode.com T: @dhelper About.ME
  • 3.
  • 4.
    [Test] public void AddItem_AddTwoValidProductsToCart_TotalEqualsItemsPrice() { varfakeRepository = A.Fake<IProductRepository>(); Product product1 = new Product(1, "product-1", 100); Product product2 = new Product(2, "product-2", 200); A.CallTo(() => fakeRepository.GetProductById(1)).Returns(product1)); A.CallTo(() => fakeRepository.GetProductById(2)).Returns(product2)); var shoppingCart = new ShoppingCart(fakeRepository); shoppingCart.AddItem(1); shoppingCart.AddItem(2); Assert.That(shoppingCart.Total, Is.EqualTo(300)); }
  • 5.
  • 6.
    A good UnitTest should be: Easy to write Simple to read Trivial to maintain
  • 7.
    What about AAA? [Test] publicasync void GetUserFromUrl() { var clientFake = A.Fake<IJsonClient>(); A.CallTo(() => clientFake.HttpGetUncompressedAsync(A<string>.Ignored)) .Returns(Task.FromResult(JsonResult)); var userRepository = new UserRepository(clientFake); var user = await userRepository.GetUser(11361); var expected = new User { Id=11361, DisplayName = "Dror Helper", ImageUrl=DefaultAvatar, Reputation=13904 }; Assert.That(user, Is.EqualTo(expected)); } Arrange  Act  Assert
  • 8.
  • 9.
    Arrange issues [TestMethod] public asyncTask LoadUser_ReputationStaysTheSame_ReputationTrendSame() { var user1 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10}; var user2 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10}; var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 }); var viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository)); await viewModel.LoadUser(); await viewModel.LoadUser(); var result = await InvokeAsync(() => ((SolidColorBrush)viewModel.ReputationTrend).Color); Assert.AreEqual(Colors.White, result); }
  • 10.
    [TestInitialize] public async TaskInitializeViewModel() { var user1 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10}; var user2 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10}; var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 }); _viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository)); } [TestMethod] public async Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() { await _viewModel.LoadUser(); await _viewModel.LoadUser(); var result = await InvokeAsync(() => ((SolidColorBrush)_viewModel.ReputationTrend).Color); Assert.AreEqual(Colors.White, result); } Using Setup/TearDown
  • 11.
    [TestInitialize] public async TaskInitializeViewModel() { var user1 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10}; var user2 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10}; var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 }); _viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository)); } [TestMethod] public async Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() { await _viewModel.LoadUser(); await _viewModel.LoadUser(); var result = await InvokeAsync(() => ((SolidColorBrush)_viewModel.ReputationTrend).Color); Assert.AreEqual(Colors.White, result); } private UserDetailsViewModel _viewModel;
  • 12.
  • 13.
    Extract to methods [TestMethod] publicasync Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() { var user1 = CreateUser(10); var user2 = CreateUser(10); var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 }); var viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository)); await viewModel.LoadUser(); await viewModel.LoadUser(); var result = await InvokeAsync(() => ((SolidColorBrush)viewModel.ReputationTrend).Color); Assert.AreEqual(Colors.White, result); }
  • 14.
    The problem withFactories private User CreateUser(int reputation) { return new User { ImageUrl = "http://dummy.jpg", Reputation = reputation }; } private User CreateUser(int reputation, string imageUrl) { return new User { ImageUrl = imageUrl, Reputation = reputation }; }
  • 15.
    Solution: Builder Pattern classUserBuilder { private int _id, _reputation; private string _name, _imageUrl; public UserBuilder() { _id = 1; _name = "dummy"; _imageUrl = "http://dummy.jpg"; } User Build() { return new User { Id = _id, Name = _name, ImageUrl = _imageUrl, Reputation = _reputation }; } }
  • 16.
    Builder Pattern (cont) classUserBuilder { ... public UserBuilder WithName(string name) { _name = name return this; } public UserBuilder WithReputation(int reputation) { _reputation = reputation return this; } ... } var user1 = new UserBuilder() .WithReputation(10) .Build();
  • 17.
    Tool: AutoMocking Containers TestContainer SUT http://blog.ploeh.dk/2013/03/11/auto-mocking-container/ New() Configure CreateCreate SUT Act
  • 18.
  • 19.
    Can you spotthe problem? [TestMethod] public void PerformSomeActionReturns42() { var myClass = ... bool initOk = myClass.Initialize(); var result = myClass.PerformSomeAction(); Assert.IsTrue(initOk); Assert.AreEqual(42, result); }
  • 20.
    How about now? [TestMethod] publicvoid TestPasswordComplexity() { var result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "1!").Result; Assert.IsFalse(result.Succeeded); result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "123456789").Result; Assert.IsFalse(result.Succeeded); result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "123456789!").Result; Assert.IsFalse(result.Succeeded); result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "abcdefghijk").Result; Assert.IsFalse(result.Succeeded); result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "abcdefghijK1!").Result; Assert.IsTrue(result.Succeeded); } http://stackoverflow.com/q/26400537/11361
  • 21.
    How many Assertsper test? One Assert per test 2 Asserts == 2 Tests Really???
  • 22.
    Multiple Asserts canmake sense [TestMethod] public void CompareTwoAsserts() { var actual = GetNextMessage(); Assert.AreEqual(1, actual.Id); Assert.AreEqual("str-1", actual.Content); }
  • 23.
    HOW UNIT TESTINGFRAMEWORKS HURT YOUR UNIT TESTING EFFORTS WHY TESTS THROW EXCEPTIONS ON FAILURE? It’s trivial, and best practice Helps Decouple test runner from test But not necessary…
  • 24.
    public class AssertAll { publicstatic void Execute(params Action[] assertionsToRun) { var errorMessages = new List<exception>(); foreach (var action in assertionsToRun) { try { action.Invoke(); } catch (Exception exc) { errorMessages.Add(exc); } } if(errorMessages.Any()) { string errorMessage = string.Join("n", errorMessages); Assert.Fail($"The following conditions failed:n{errorMessage}")); } } } https://helpercode.com/2011/02/16/multiple-asserts-done-right/
  • 25.
    Using Assert.All [TestMethod] public voidCompareTwoAsserts() { var actual = CreateMessage(); AssertAll.Execute( () => Assert.AreEqual(1, actual.Id), () => Assert.AreEqual("str-1", actual.Content); }
  • 26.
    Some frameworks arecatching up! https://github.com/nunit/docs/wiki/Multiple-Asserts
  • 27.
    Assert.AreEqual(5, 2 +2); Assert.AreEqual(2 + 2, 5)
  • 28.
    Assert.AreEqual(5, 2 +2); Assert.IsTrue(2 + 2 == 5)
  • 29.
    How to choosethe right Assert? • IsTrue vs. AreEqual • Parameter ordering confusion • StringAssert/CollectionAssert It’s all about proper error messages
  • 30.
    AssertHelper [Test] public void CompareTest() { varmyClass = new MyClass(); Expect.That(() => myClass.ReturnFive() == 10); } [Test] public void CompareTest() { var c1 = new[] { 1, 2, 3, 4, 5 } Expect.That(() => c1.Contains(42); } https://github.com/dhelper/AssertHelper
  • 31.
    Assertion Libraries: FluentAssertions [Fact] publicvoid CompareTwoObjects() { var customer1 = new Customer("cust-1", "John Doe"); var customer2 = new Customer("cust-2", "John Doe"); customer1.ShouldBeEquivalentTo(customer2, o => o.Excluding(customer => customer.Id)); } http://www.fluentassertions.com/
  • 32.
  • 33.
    Test structure issues •What to call the test? • AAA is not mandatory • What should I test? • How to avoid unreadable, complicated tests? - Unit testing frameworks do not provide structure
  • 34.
  • 35.
  • 36.
    [TestClass] public class CardHasBeenDisabled{ private Card _card; private Atm _subject; void GivenTheCardIsDisabled() { _card = new Card(false, 100); _subject = new Atm(100); } void WhenTheAccountHolderRequestsMoney() { _subject.RequestMoney(_card, 20); } void ThenTheAtmShouldRetainTheCard() { Assert.IsTrue(_subject.CardIsRetained); } void AndTheAtmShouldSayTheCardHasBeenRetained() { Assert.AreEqual(DisplayMessage.CardIsRetained, _subject.Message); } [TestMethod] public void Execute() { this.BDDfy(); } } Tool: BDDfy
  • 37.
  • 38.
  • 39.
    The right toolswill help you write good tests Arrange Builder Pattern AutoMocking Containers Assert Multiple Asserts 3rd party Assertion libraries Test Structure “Classic” BDD In-Code BDD Continuous Testing Typemock Runner DotCover nCrunch Live Unit Testing Corvlet (.NET Core)
  • 40.
    Thank you @dhelper |http://helpercode.com

Editor's Notes

  • #18 SHOW DEMO!
  • #30 - Two asserts – no idea what caused the failure Test is testing several things Left or right? InstanceOf MSTest vs. NUnit