Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Secret unit testing tools no one ever told you about

265 views

Published on

There are more to unit testing than using a unit testing framework – in order to succeed you want to use the right tools for the job. There are a few tools that almost no one talks about – some enabling creating of top-notch, robust unit tests. Some will help you run your tests better and faster.
In this session I’ll explain about the inevitable maintainability problems developers face when writing and maintaining huge unit testing suits and how unit level BDD, AutoMocking, and Continuous Execution can help take control over your tests.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Secret unit testing tools no one ever told you about

  1. 1. The secret unit testing tools no one ever told you about Dror Helper (@dhelper) Demos: https://github.com/dhelper/SecretUnitTestingToolsDemos
  2. 2. About.ME Consultant @CodeValue Developing software (professionally) since 2002 Mocking code since 2008 Clean coder & Test Driven Developer Pluralsight Author Blogger: http://blog.drorhelper.com
  3. 3. Why should I care about tools?
  4. 4. Background: unit testing tools
  5. 5. Well known unit testing tools
  6. 6. Server Dev Machine Source ControlBuild Server Test Runner Code Coverage Build Agent Unit Testing Framework Isolation Framework ?
  7. 7. xUnit test framework public class BeforeAndAfter { [SetUp] public void Initialize() { } [TearDown] public void Cleanup() { } [Test] public void test1() { } [Test] public void test2() { } } Test Suite Fixture Test Case Test Case Test Case Test Case Test Case Fixture Test Case Test Case Fixture Test Case Test Case Test Case
  8. 8. Mocking Frameworks Unit test Code under test DependencyFake object(s)
  9. 9. What Mocking framework do? 1. Create Fake objects 2. Set behavior on fake objects 3. Verify method was called And more...
  10. 10. [Test] public void Calculate_ReturnTwoValidNumbers_ServerCalled() { IDataAccess fakeDataAccess = A.Fake<IDataAccess>(); A.CallTo(() => fakeDataAccess.GetData(A<string>.Ignored)) .Returns(new Tuple<int, int>(2, 3)); var fakeCalculatorService = A.Fake<ICalculatorService>(); var cut = new DistrobutedCalculator(fakeDataAccess, fakeCalculatorService); cut.Calculate(); A.CallTo(() => fakeCalculatorService.Add(2,3)).MustHaveHappened(); }
  11. 11. Definition: unit tests Automated piece of code that invokes a unit of work in the system and then checks a single assumption about the behavior of that unit of work [Roy Osherove, The Art Of Unit Testing]
  12. 12. Unit test structure [Test] public void MyTest() { }
  13. 13. No guidance  Fragile tests  Stop unit testing
  14. 14. What about AAA? [Test] public async void GetUserFromUrl() { var clientFake = A.Fake<IJsonClient>(); A.CallTo(() => clientFake.HttpGetUncompressedAsync(A<string>.Ignored)) .Returns(Task.FromResult(JsonResult)); var userRepository = new UserRepository(clientFake); var user = await userRepository.GetUser(11361); var expected = new User { Id=11361, DisplayName = "Dror Helper", ImageUrl=DefaultAvatar, Reputation=13904 }; Assert.That(user, Is.EqualTo(expected)); }  Act  Assert Arrange
  15. 15. Test setup (Arrange)
  16. 16. Arrange issues [TestMethod] public async Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() { var user1 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10}; var user2 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10}; var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 }); var viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository)); await viewModel.LoadUser(); await viewModel.LoadUser(); var result = await InvokeAsync(() => ((SolidColorBrush)viewModel.ReputationTrend).Color); Assert.AreEqual(Colors.White, result); }
  17. 17. Solution: Setup/TearDown [TestInitialize] public async Task InitilizeUserViewModel() { var user1 = new User { ImageUrl = "http://dummy.jpg", Reputation = 10 }; var user2 = new User { ImageUrl = "http://dummy.jpg", Reputation = 10 }; var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 }); _viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository)); } [TestMethod] public async Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() { await _viewModel.LoadUser(); await _viewModel.LoadUser(); var result = InvokeAsync(() => ((SolidColorBrush)_viewModel.ReputationTrend).Color); Assert.AreEqual(Colors.White, result); }
  18. 18. You shouldn’t use Setup in unit tests private UserDetailsViewModel _viewModel; [TestInitialize] public async Task InitilizeUserViewModel() { var user1 = new User { ImageUrl = "http://dummy.jpg", Reputation = 10 }; var user2 = new User { ImageUrl = "http://dummy.jpg", Reputation = 10 }; var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 }); _viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository)); } [TestMethod] public async Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() { await _viewModel.LoadUser(); await _viewModel.LoadUser(); var result = InvokeAsync(() => ((SolidColorBrush)_viewModel.ReputationTrend).Color); Assert.AreEqual(Colors.White, result); }
  19. 19. Solution: Extract to methods [TestMethod] public async Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() { var user1 = CreateUser(reputation: 10); var user2 = CreateUser(reputation: 10); var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 }); var viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository)); await viewModel.LoadUser(); await viewModel.LoadUser(); var result = await InvokeAsync(() => ((SolidColorBrush)viewModel.ReputationTrend).Color); Assert.AreEqual(Colors.White, result); }
  20. 20. Over extraction [TestMethod] public async Task LoadUser_ReputationStaysTheSame() { var viewModel = InitializeSystem(10, 10); await viewModel.LoadUser(); await viewModel.LoadUser(); CheckColor(Colors.White, viewModel); }
  21. 21. The problem with factories private User CreateUser(int reputation) { return new User { ImageUrl = "http://dummy.jpg", Reputation = reputation }; } private User CreateUser(int reputation, string imageUrl) { return new User { ImageUrl = imageUrl, Reputation = reputation }; }
  22. 22. Builder pattern class UserBuilder { private int _id, _reputation; private string _name, _imageUrl; public UserBuilder() { _id = 1; _name = "dummy"; _imageUrl = "http://dummy.jpg"; } User Build() { return new User { Id = _id, Name = _name, ImageUrl = _imageUrl, Reputation = _reputation }; } }
  23. 23. Builder pattern (cont.) class UserBuilder { ... public UserBuilder WithName(string displayName) { _displayName = displayName; return this; } public UserBuilder WithReputation(int reputation) { _reputation = reputation; return this; } ... } var user1 = new UserBuilder() .WithReputation(10) .Build();
  24. 24. Tool: AutoMocking Containers Test Container SUT http://blog.ploeh.dk/2013/03/11/auto-mocking-container/ New() Configure CreateCreate SUT Act
  25. 25. Automocking with AutoFixture [Fact] public void YellIfTouchHotIron() { var fixture = new Fixture().Customize(new AutoFakeItEasyCustomization()); Fake<IMouth> fakeMouth = fixture.Freeze<Fake<IMouth>>(); Fake<IHand> fakeHand = fixture.Freeze<Fake<IHand>>(); A.CallTo(() => fakeHand.FakedObject.TouchIron(A<Iron>._)).Throws<BurnException>(); var brain = fixture.Create<Brain>(); brain.TouchIron(new Iron {IsHot = true}); A.CallTo(() => fakeMouth.FakedObject.Yell()).MustHaveHappened(); } https://helpercode.com/2013/12/22/on-object-creation-and-unit-tests/
  26. 26. Problem: Complex inputs • Big objects • Deep objects • Need for precision • Lack of knowledge
  27. 27. Solution: use trivial inputs Sometimes missing the point Not always enough for “real test”
  28. 28. Solution: Serialization Supported in most programming languages (XML, json). Need development  testing delayed Production code change  indefinitely delayed
  29. 29. Tool: Export using OzCode
  30. 30. Verification (Assert)
  31. 31. Can you spot the problem? [TestMethod] public void PerformSomeActionReturns42() { var myClass = ... bool initOk = myClass.Initialize(); var result = myClass.PerformSomeAction(); Assert.IsTrue(initOk); Assert.AreEqual(42, result); } http://stackoverflow.com/q/26400537/11361
  32. 32. Can you spot the problem? [TestMethod] public void TestPasswordComplexity() { var result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "1!").Result; //Changes the password. Assert.IsFalse(result.Succeeded); result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "123456789").Result; //Changes the password. Assert.IsFalse(result.Succeeded); result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "123456789!").Result; //Changes the password. Assert.IsFalse(result.Succeeded); result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "abcdefghijk").Result; //Changes the password. Assert.IsFalse(result.Succeeded); result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "abcdefghijK1!").Result; //Changes the password. Assert.IsTrue(result.Succeeded); } http://stackoverflow.com/q/26400537/11361
  33. 33. How many Assert(s) per test? One Assert Per Test! Two Assert == Two Tests  Usually ??? ”(…the Code is more) what you'd call guidelines than actual rules”
  34. 34. Sometimes multiple asserts make sense [TestMethod] public void CompareTwoAsserts() { var actual = GetNextMessage(); Assert.AreEqual(1, actual.Id); Assert.AreEqual("str-1", actual.Content); }
  35. 35. public class AssertAll { public static void Execute(params Action[] assertionsToRun) { var errorMessages = new List<exception>(); foreach (var action in assertionsToRun) { try { action.Invoke(); } catch (Exception exc) { errorMessages.Add(exc); } } if(errorMessages.Any()) { var separator = string.Format("{0}{0}", Environment.NewLine); string errorMessage = string.Join(separator, errorMessages); Assert.Fail(string.Format("The following conditions failed:{0}{1}", Environment.NewLine, errorMessage)); } } } http://blog.drorhelper.com/2011/02/multiple-asserts-done-right.html
  36. 36. Using AssertAll [TestMethod] public void CompareTwoAsserts() { var actual = CreateMessage(); AssertAll.Execute( () => Assert.AreEqual(1, actual.Id), () => Assert.AreEqual("str-1", actual.Content); }
  37. 37. Some frameworks are catching up! https://github.com/nunit/docs/wiki/Multiple-Asserts
  38. 38. Choosing the right Assert? IsTrue vs. AreEqual Parameter ordering confusion StringAssert/CollectionAssert It’s all about proper error messages
  39. 39. Tool: 3rd party assertion libraries Better error messages Readability Multiple asserts* ×Additional dependency ×Limited UT framework support ×System.Object “SPAMED” by extension methods
  40. 40. Shouldly [Fact] public void AddTest() { var calculator = new Calculator(); var result = calculator.Add(2, 3); Assert.Equal(6, result); } [Fact] public void AddTest_Shouldly() { var calculator = new Calculator(); var result = calculator.Add(2, 3); result.ShouldBe(6); } https://github.com/shouldly/shouldly Shouldly.ShouldAssertException result should be 6 but was 5 Xunit.Sdk.EqualException Assert.Equal() Failure Expected: 6 Actual: 5
  41. 41. FluentAssertions [Fact] public void CompareTwoObjects() { var customer1 = new Customer("cust-1", "John Doe"); var customer2 = new Customer("cust-2", "John Doe"); customer1.ShouldBeEquivalentTo(customer2, o => o.Excluding(customer => customer.Id)); } http://www.fluentassertions.com/
  42. 42. AssertHelper [Test] public void CheckCompare() { var myClass = new MyClass(); Expect.That(() => myClass.ReturnFive() == 10); } [Test] public void CheckTrue() { var myClass = new MyClass(); Expect.That(() => myClass.ReturnFalse() == true); } [Test] public void StringStartsWith() { var s1 = "1234567890"; Expect.That(() => s1.StartsWith("456")); } [Test] public void CollectionContains() { var c1 = new[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0 }; Expect.That(() => c1.Contains(41)); } https://github.com/dhelper/AssertHelper
  43. 43. Test organization
  44. 44. Test structure issues What to call the test? AAA is not mandatory What should I test? How to avoid unreadable, complicated tests? - Unit testing framework provide no structure
  45. 45. The BDD approach Step Definitions
  46. 46. Specifications == focused test Feature: Addition In order to avoid silly mistakes As a math idiot I want to be told the sum of two numbers Scenario: Add two numbers Given I have entered 50 into the calculator And I have entered 70 into the calculator When I press add Then the result should be 120 on the screen
  47. 47. BDD Example: SpecFlow http://www.specflow.org/
  48. 48. Tool: BDDfy [TestClass] public class CardHasBeenDisabled { private Card _card; private Atm _subject; void GivenTheCardIsDisabled() { _card = new Card(false, 100); _subject = new Atm(100); } void WhenTheAccountHolderRequestsMoney() { _subject.RequestMoney(_card, 20); } void ThenTheAtmShouldRetainTheCard() { Assert.IsTrue(_subject.CardIsRetained); } void AndTheAtmShouldSayTheCardHasBeenRetained() { Assert.AreEqual(DisplayMessage.CardIsRetained, _subject.Message); } [TestMethod] public void Execute() { this.BDDfy(); } }
  49. 49. Test execution
  50. 50. Everybody needs a CI server Unit tests without a CI server are a waste of time - if you're running all of the tests all of the time locally you're a better man then I am
  51. 51. Tool: Continuous testing DotCover Typemock Runner nCrunch VS2017 (Enterprise)
  52. 52. The right tools will help you write good tests Arrange Builder Pattern AutoMocking Containers Export Assert Shouldly FluentAssertions AssertHelper Test Structure BDDfy Continuous Testing Typemock Runner DotCover nCrunch Live Unit Testing
  53. 53. Dror Helper | @ dhelper | http://helpercode.com Thank you! Demos: https://github.com/dhelper/SecretUnitTestingToolsDemos

×