Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Secret unit testing tools

383 views

Published on

There is more to unit testing than using a unit testing framework. In order to succeed you want to use the right tools for the job. There are a few tools that almost no one talks about – some enabling the creation of top-notch, robust unit tests; some will help you run your tests better and faster.

In this session, Dror will explain about the inevitable maintainability problems developers face when writing and maintaining huge unit testing suits, and how unit level BDD, AutoMocking and Continuous Execution can help take control over your tests.

Published in: Software
  • Be the first to comment

Secret unit testing tools

  1. 1. The secret unit testing tools no one has ever told you about Dror Helper | blog.drorhelper.com | @dhelper Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek
  2. 2. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek About.ME Consultant @CodeValue Developing software (professionally) since 2002 Mocking code since 2008 Clean coder & Test Driven Developer OzCode (a.k.a “Magical debugging”) Evangelist Blogger: http://blog.drorhelper.com
  3. 3. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek But it was not always like that 1st Attempt Failed! 2nd Attempt Failed!!! New job + UT + Mentor Success
  4. 4. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Why should I care about tools?
  5. 5. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Background: unit testing tools
  6. 6. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Well known unit testing tools Build Failed!
  7. 7. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Server Dev Machine Source ControlBuild Server Test Runner Code Coverage Build Agent Unit Testing Framework Isolation Framework ?
  8. 8. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek xUnit test framework Test Suite Fixture Test Case Test Case Test Case Test Case Test Case Fixture Test Case Test Case Fixture Test Case Test Case Test Case public class BeforeAndAfter { [SetUp] public void Initialize() { } [TearDown] public void Cleanup() { } [Test] public void test1() { } [Test] public void test2() { } }
  9. 9. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Mocking Frameworks Unit test Code under test DependencyFake object(s)
  10. 10. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek What Mocking framework can do for you? • Create Fake objects • Set behavior on fake objects • Verify method was called • And more...
  11. 11. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek [Test] public void Calculate_ReturnTwoValidNumbers_ServerCalled() { IDataAccess fakeDataAccess = A.Fake<IDataAccess>(); A.CallTo(() => fakeDataAccess.GetData(A<string>.Ignored)) .Returns(new Tuple<int, int>(2, 3)); var fakeCalculatorService = A.Fake<ICalculatorService>(); var cut = new DistrobutedCalculator(fakeDataAccess, fakeCalculatorService); cut.Calculate(); A.CallTo(() => fakeCalculatorService.Add(2,3)).MustHaveHappened(); }
  12. 12. These tools do not help us write good unit tests In fact, sometimes they prevent us from writing good unit tests!
  13. 13. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Definition: unit tests Automated piece of code that invokes a unit of work in the system and then checks a single assumption about the behavior of that unit of work [Roy Osherove, The Art Of Unit Testing]
  14. 14. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Unit test structure [Test] public void MyTest() { }
  15. 15. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek No guidance  Fragile tests  Stop unit testing
  16. 16. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek What about AAA? [Test] public async void GetUserFromUrl() { var clientFake = A.Fake<IJsonClient>(); A.CallTo(() => clientFake.HttpGetUncompressedAsync(A<string>.Ignored)) .Returns(Task.FromResult(JsonResult)); var userRepository = new UserRepository(clientFake); var user = await userRepository.GetUser(11361); var expected = new User { Id=11361, DisplayName = "Dror Helper", ImageUrl=DefaultAvatar, Reputation=13904 }; Assert.That(user, Is.EqualTo(expected)); } Arrange  Act  Assert
  17. 17. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Problem solved? Hardly! • Where to start? • How to test existing code? • What about test structure? • Integration tests vs. unit tests • What is a “unit of work”?
  18. 18. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Test setup (Arrange)
  19. 19. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Arrange issues [TestMethod] public async Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() { var user1 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10}; var user2 = new User {ImageUrl = "http://dummy.jpg", Reputation = 10}; var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 }); var viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository)); await viewModel.LoadUser(); await viewModel.LoadUser(); var result = await InvokeAsync(() => ((SolidColorBrush)viewModel.ReputationTrend).Color); Assert.AreEqual(Colors.White, result); }
  20. 20. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Solution: Setup/TearDown private UserDetailsViewModel _viewModel; [TestInitialize] public async Task InitilizeUserViewModel() { var user1 = new User { ImageUrl = "http://dummy.jpg", Reputation = 10 }; var user2 = new User { ImageUrl = "http://dummy.jpg", Reputation = 10 }; var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 }); _viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository)); } [TestMethod] public async Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() { await _viewModel.LoadUser(); await _viewModel.LoadUser(); var result = InvokeAsync(() => ((SolidColorBrush)_viewModel.ReputationTrend).Color); Assert.AreEqual(Colors.White, result); }
  21. 21. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Why you shouldn’t use Setup in unit tests private UserDetailsViewModel _viewModel; [TestInitialize] public async Task InitilizeUserViewModel() { var user1 = new User { ImageUrl = "http://dummy.jpg", Reputation = 10 }; var user2 = new User { ImageUrl = "http://dummy.jpg", Reputation = 10 }; var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 }); _viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository)); } [TestMethod] public async Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() { await _viewModel.LoadUser(); await _viewModel.LoadUser(); var result = InvokeAsync(() => ((SolidColorBrush)_viewModel.ReputationTrend).Color); Assert.AreEqual(Colors.White, result); }
  22. 22. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Solution: Extract to methods [TestMethod] public async Task LoadUser_ReputationStaysTheSame_ReputationTrendSame() { var user1 = CreateUser(reputation: 10); var user2 = CreateUser(reputation: 10); var fakeUserRepository = new FakeUserRepository(new[] { user1, user2 }); var viewModel = await InvokeAsync(() => new UserDetailsViewModel(fakeUserRepository)); await viewModel.LoadUser(); await viewModel.LoadUser(); var result = await InvokeAsync(() => ((SolidColorBrush)viewModel.ReputationTrend).Color); Assert.AreEqual(Colors.White, result); }
  23. 23. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Over extraction [TestMethod] public async Task LoadUser_ReputationStaysTheSame() { var viewModel = InitializeSystem(10, 10); await viewModel.LoadUser(); await viewModel.LoadUser(); CheckColor(Colors.White, viewModel); }
  24. 24. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek The problem with factories private User CreateUser(int reputation) { return new User { ImageUrl = "http://dummy.jpg", Reputation = reputation }; } private User CreateUser(int reputation, string imageUrl) { return new User { ImageUrl = imageUrl, Reputation = reputation }; }
  25. 25. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Builder pattern class UserBuilder { private int _id, _reputation; private string _displayName, _imageUrl; public UserBuilder() { _id = 1; _displayName = "dummy"; _imageUrl = "http://dummy.jpg"; } User Build() { return new User { Id = _id, DisplayName = _displayName, ImageUrl = _imageUrl, Reputation = _reputation }; } }
  26. 26. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Builder pattern (cont.) class UserBuilder { ... public UserBuilder WithName(string displayName) { _displayName = displayName; return this; } public UserBuilder WithReputation(int reputation) { _reputation = reputation; return this; } ... } var user1 = new UserBuilder() .WithReputation(10) .Build();
  27. 27. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Tool: AutoMocking Containers Test Container SUT http://blog.ploeh.dk/2013/03/11/auto-mocking-container/ New() Configure CreateCreate SUT Act
  28. 28. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Automocking with AutoFixture [Fact] public void YellIfTouchHotIron() { var fixture = new Fixture().Customize(new AutoFakeItEasyCustomization()); Fake<IMouth> fakeMouth = fixture.Freeze<Fake<IMouth>>(); Fake<IHand> fakeHand = fixture.Freeze<Fake<IHand>>(); A.CallTo(() => fakeHand.FakedObject.TouchIron(A<Iron>._)).Throws<BurnException>(); var brain = fixture.Create<Brain>(); brain.TouchIron(new Iron {IsHot = true}); A.CallTo(() => fakeMouth.FakedObject.Yell()).MustHaveHappened(); }
  29. 29. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Automocking with Typemock Isolator [TestMethod] public void FakeAllDependencies_ChangeBehavior() { var real = Isolate.Fake.Dependencies<ClassUnderTest>(); var fake = Isolate.GetFake<Dependency>(real); Isolate.WhenCalled(() => fake.Multiplier).WillReturn(2); var result = real.Calculate(1, 2); Assert.AreEqual(6, result); }
  30. 30. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Problem: Complex inputs • Big objects • Deep objects • Need for precision • Lack of knowledge
  31. 31. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Solution: use trivial inputs Sometimes missing the point Not always enough for “real test”
  32. 32. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Solution: Serialization Supported in most programming languages (XML, json). • Need development  testing delayed • Production code change  indefinitely delayed
  33. 33. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Tool: Export using OzCode
  34. 34. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Verification (Assert)
  35. 35. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Can you spot the problem? [TestMethod] public void PerformSomeActionReturns42() { var myClass = ... bool initOk = myClass.Initialize(); var result = myClass.PerformSomeAction(); Assert.IsTrue(initOk); Assert.AreEqual(42, result); } http://stackoverflow.com/q/26400537/11361
  36. 36. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Can you spot the problem? [TestMethod] public void TestPasswordComplexity() { var result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "1!").Result; //Changes the password. Assert.IsFalse(result.Succeeded); result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "123456789").Result; //Changes the password. Assert.IsFalse(result.Succeeded); result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "123456789!").Result; //Changes the password. Assert.IsFalse(result.Succeeded); result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "abcdefghijk").Result; //Changes the password. Assert.IsFalse(result.Succeeded); result = _UserManager.ChangePasswordAsync(_TestUser.Id, "Password123!", "abcdefghijK1!").Result; //Changes the password. Assert.IsTrue(result.Succeeded); } http://stackoverflow.com/q/26400537/11361
  37. 37. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek How many Assert(s) per test? One Assert Per Test! Two Assert == Two Tests  Usually ??? ”(…the Code is more) what you'd call guidelines than actual rules”
  38. 38. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Sometimes multiple asserts make sense [TestMethod] public void CompareTwoAsserts() { var actual = GetNextMessage(); Assert.AreEqual(1, actual.Id); Assert.AreEqual("str-1", actual.Content); }
  39. 39. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek public class AssertAll { public static void Execute(params Action[] assertionsToRun) { var errorMessages = new List<exception>(); foreach (var action in assertionsToRun) { try { action.Invoke(); } catch (Exception exc) { errorMessages.Add(exc); } } if(errorMessages.Any()) { var separator = string.Format("{0}{0}", Environment.NewLine); string errorMessage = string.Join(separator, errorMessages); Assert.Fail(string.Format("The following conditions failed:{0}{1}", Environment.NewLine, errorMessage)); } } } http://blog.drorhelper.com/2011/02/multiple-asserts-done-right.html
  40. 40. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Using AssertAll [TestMethod] public void CompareTwoAsserts() { var actual = CreateMessage(); AssertAll.Execute( () => Assert.AreEqual(1, actual.Id), () => Assert.AreEqual("str-1", actual.Content); }
  41. 41. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Some frameworks are catching up! https://github.com/nunit/docs/wiki/Multiple-Asserts-Spec
  42. 42. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Ever had issues choosing the right Assert? • IsTrue vs. AreEqual • Parameter ordering confusion • StringAssert/CollectionAssert It’s all about proper error messages
  43. 43. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Tool: 3rd party assertion libraries Better error messages Readability Multiple asserts* ×Additional dependency ×Limited UT framework support ×System.Object “SPAMED” by extension messages
  44. 44. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Shouldly [Fact] public void AddTest() { var calculator = new Calculator(); var result = calculator.Add(2, 3); Assert.Equal(6, result); } [Fact] public void AddTest_Shouldly() { var calculator = new Calculator(); var result = calculator.Add(2, 3); result.ShouldBe(6); } https://github.com/shouldly/shouldly Shouldly.ShouldAssertException result should be 6 but was 5 Xunit.Sdk.EqualException Assert.Equal() Failure Expected: 6 Actual: 5
  45. 45. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Shouldly [Fact] public void GetDivisorsTest() { var calculator = new Calculator(); var result = calculator.GetDivisors(20); Assert.Equal(new[] {2,3,5,7}, result); } [Fact] public void GetDivisorsTest_Shouldly() { var calculator = new Calculator(); var result = calculator.GetDivisors(20); result.ShouldBe(new[] { 2, 3, 5, 7 }); } https://github.com/shouldly/shouldly Shouldly.ShouldAssertException result should be [2, 3, 5, 7] but was [2, 4, 5, 10] difference [2, *4*, 5, *10*] Xunit.Sdk.EqualException Assert.Equal() Failure Expected: Int32[] [2, 3, 5, 7] Actual: WhereEnumerableIterator<Int32> [2, 4, 5, 10]
  46. 46. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek FluentAssertions [Fact] public void CompareTwoObjects() { var customer1 = new Customer("cust-1", "John Doe"); var customer2 = new Customer("cust-2", "John Doe"); customer1.ShouldBeEquivalentTo(customer2, o => o.Excluding(customer => customer.Id)); } http://www.fluentassertions.com/
  47. 47. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek AssertHelper [Test] public void CheckCompare() { var myClass = new MyClass(); Expect.That(() => myClass.ReturnFive() == 10); } [Test] public void CheckTrue() { var myClass = new MyClass(); Expect.That(() => myClass.ReturnFalse() == true); } [Test] public void StringStartsWith() { var s1 = "1234567890"; Expect.That(() => s1.StartsWith("456")); } [Test] public void CollectionContains() { var c1 = new[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0 }; Expect.That(() => c1.Contains(41)); } https://github.com/dhelper/AssertHelper
  48. 48. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Test organization
  49. 49. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Test structure issues • What to call the test? • AAA is not mandatory • What should I test? • How to avoid unreadable, complicated tests? - Unit testing framework provide no structure
  50. 50. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek The BDD approach Step Definitions
  51. 51. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Specifications == focused test Feature: Addition In order to avoid silly mistakes As a math idiot I want to be told the sum of two numbers Scenario: Add two numbers Given I have entered 50 into the calculator And I have entered 70 into the calculator When I press add Then the result should be 120 on the screen
  52. 52. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek BDD Example: SpecFlow http://www.specflow.org/
  53. 53. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Tool: BDDfy [TestClass] public class CardHasBeenDisabled { private Card _card; private Atm _subject; void GivenTheCardIsDisabled() { _card = new Card(false, 100); _subject = new Atm(100); } void WhenTheAccountHolderRequestsMoney() { _subject.RequestMoney(_card, 20); } void ThenTheAtmShouldRetainTheCard() { Assert.IsTrue(_subject.CardIsRetained); } void AndTheAtmShouldSayTheCardHasBeenRetained() { Assert.AreEqual(DisplayMessage.CardIsRetained, _subject.Message); } [TestMethod] public void Execute() { this.BDDfy(); } }
  54. 54. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Test execution
  55. 55. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Everybody needs a CI server Unit tests without a CI server are a waste of time - if you're running all of the tests all of the time locally you're a better man then I am
  56. 56. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Tool: Continuous testing DotCover Typemock Runner nCrunch
  57. 57. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek The right tools will help you write good tests Arrange Builder Pattern AutoMocking Containers Export Assert Shouldly FluentAssertions AssertHelper Test Structure BDDfy Continuous Testing Typemock Runner DotCover nCrunch
  58. 58. Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek Thank you Dror Helper | @dhelper | http://blog.drorhelper.com

×