The Evolution of Development Testing

  • 141 views
Uploaded on

A look at concepts, practices and tools related to Java test code readability and maintainability

A look at concepts, practices and tools related to Java test code readability and maintainability

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
141
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
3
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The Evolution of Development Testing
  • 2. Overview • Automating • Testing Categories + Scope • Development Testing • Principles, Practices + Tool Support
  • 3. Overview • Automating
  • 4. Automating • We automate “stuff” for a living • Developer/Programmer/Engineer/Automator • When we’re doing things manually we’re usually missing an opportunity to automate
  • 5. Automating • The stuff we automate might be • validation of a web form • reporting of a message parsing error • calculation of an average execution price • execution + status reporting of tests for all of the above
  • 6. Automating • That automation might look like • choose validation rules, implement, enforce, report • log parsing error, automate monitoring/reporting of error • define calculation rules, implement, provide result • define inputs, context + expectations, execute scenario, determine + report status
  • 7. Automating • Manual solutions might look like • review user errors later in workflow e.g.“back office” manual activity • message loss caused an incident, manually track it down • export raw data to csv, load into Excel, apply formula • run UI functional scenarios that use/execute the code in mind • fire up debugger, step through mis-behaving code • add detailed logging, monitor logs for hints
  • 8. Automating • We automate “stuff” for a living • Developer/Programmer/Engineer/Automator • When we’re doing things manually we’re usually missing an opportunity to automate
  • 9. Overview • Automating • Testing Categories + Scope
  • 10. Test Categorisation • Useful concept - up to a point • Blurred lines + Overloaded terminology • Difficulties conversing about testing
  • 11. Test Categorisation • Unit • Build Acceptance • Component • System • API • End-to-End • Acceptance • User Acceptance • Integration • Non-functional • Functional
  • 12. Test Categorisation
  • 13. Test Categorisation
  • 14. Test Categorisation • Useful concept - up to a point • Blurred lines + Overloaded terminology • Difficulties conversing about testing
  • 15. Test Categorisation
  • 16. Test Categorisation
  • 17. Overview • Automating • Testing Categories + Scope • Development Testing
  • 18. Development Testing • What it is • What it should not be • How it has evolved
  • 19. What it is • A Development practice first and foremost • White box / Black box / Grey box • Has Quality aims • Has Efficiency aims
  • 20. JUnit • Test automation framework of choice for most Java developers • Synonymous with Unit testing • Widely used for testing beyond the Unit scope • Ported to many languages - generically known as xUnit
  • 21. JUnit “Never in the field of software development have so many owed so much to so few lines of code” Martin Fowler (Development big-wig)
  • 22. What it is • A Development practice first and foremost • White box / Black box / Grey box • Has Quality aims • Has Efficiency aims
  • 23. What it is • Can take a technical perspective on scenarios • Can take a user functionality perspective • Exercises precise control over the test context • Used + extended by Dev during initial and future coding
  • 24. What it is • Often used to discover and refine solution design • Often leaves fine-grained regression coverage safety net • Enables instant feedback on failures due to future enhancements and refactorings
  • 25. What it is not • Automated full System tests a.k.a E2E, System Integration • Zero control over the state of the system (the test context) • Test scope coupled to other external systems, reference data quality, env availability • Heavily reliant on driving a UI to execute many finegrained scenarios
  • 26. Its Evolution • OOP - slow evolution • Automated Developer Testing around a lot less time • A lot less mainstream than OOP • Growing body of knowledge / experience available
  • 27. Overview • Automating • Testing Categories + Scope • Development Testing • Principles, Practices + Tool Support
  • 28. Principles, Practices + Tool Support • Structural Principles / Characteristics • Qualities of a “good” test / spec • Practices + Tools for improving test qualities
  • 29. Test structure • Arrange, Act, Assert • Specify Inputs • Given, When, Then • Specify the state of the System • Specify the Event • Specify the Expectations
  • 30. Test structure • Naming conventions + structure @Test public // // // } void methodUnderTest_GivenABC_ThenExpectXYZ() { given when then @Test public void methodUnderTest_GivenInputsABC_AndSystemStateDEF_ThenExpectXYZ() { // given - inputs // given - system state ! // when // then }
  • 31. Test structure • IDE assistance for consistency + evolve conventions
  • 32. Principles, Practices + Tool Support • Structural Principles / Characteristics • Qualities of a “good” test / spec
  • 33. “Any fool can write code that a computer can understand” ! Martin Fowler
  • 34. Test Qualities • Readability • What is it? Why does it matter?
  • 35. Test Qualities • Readability == …? • Code that you can understand with ease • Language affects Thought • Thought affects Action
  • 36. Test Qualities • Tools can directly address Readability qualities • Domain Specific Languages = a fancy term • • JUnit, Hamcrest, FEST, JMock, Mockito, Gherkin Language -> Thought -> Action
  • 37. “Test automation is a first class software engineering problem” ! Brian Marick?
  • 38. Test Qualities • Maintainability • What is it? Why does it matter?
  • 39. Test Qualities • Maintainability == …? • To change, evolve with ease • Keeping the cost of change down
  • 40. Test Qualities • They can be context sensitive. Many are not. • You may opt to sacrifice: • • Readability for Maintainability • • DRY Principle for Readability Maintainability for Obviousness Expect debate
  • 41. Principles, Practices + Tool Support • Structural Principles / Characteristics • Qualities of a “good” test / spec • Practices + Tools for improving test qualities
  • 42. Specifying Expectations • Stating Expectations / Asking Questions • From Computer dialect ——> Human dialect // then assertTrue(trader.getPermissions().isEmpty()); assertThat(trader.getPermissions().isEmpty(), is(true)); assertThat(trader.getPermissions()).isEmpty(); assertThat(trader).hasNoPermissions(); // // // // // // // Computer dialect . . . . . Human dialect
  • 43. Specifying Expectations • Stating Expectations / Asking Questions • From Computer dialect ——> Human dialect // Using Basic JUnit assertEquals(orders.size(), 2); Iterator<Order> itr = orders.iterator(); assertEquals(itr.next(), order3); assertEquals(itr.next(), order4); // Using Hamcrest assertThat(orders, hasItems(order3, order4)); assertThat(orders.size(), is(2)); // Using FEST Assertions assertThat(orders).containsOnly(order3, order4); // Using FEST Assertions (chained assertions) assertThat(orders).containsOnly(order3, order4) .containsSequence(order3, order4);
  • 44. Specifying Expectations • Easier + Faster to write expressive statements • Faster to read + review • Easier to maintain • Encourages other better practices
  • 45. Specifying Inputs • Trivial example. Test with very narrow scope • e.g.Input = Single Object @Test public void getPermissions_GivenNewUser_ThenReturnsNoPermissions() { // given Trader trader = new Trader(); // when + then assertThat(trader.getPermissions()).isEmpty(); }
  • 46. Specifying Inputs • Non-trivial example. Test with broader scope • e.g. Input = Object Graph • i.e. objects linked to objects
  • 47. @Test public void getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v1() { // given - inputs Trader trader = new Trader(); TradingAccount tradingAcct = new TradingAccount(new TradingFirm("TraderzRUs", "TRDRZ"), new ClearingFirm("TooBig2Fail", "2BIG2F"), ACTIVE); trader.setPermissions(Arrays.asList(new Permission(tradingAcct))); String goldIsinCode = "COF24680"; // given - system state Future oil3MnthFuture = new Future("OIL.3MNTH", new ISIN("OIL3M0123")); Future oil6MnthFuture = new Future("OIL.6MNTH", new ISIN("OIL6M0456")); Future goldFuture = new Future("GLD.3MNTH", new ISIN(goldIsinCode)); String orderId = "ordId"; Order order1 = new Order(orderId + 1, tradingAcct, oil3MnthFuture, 1000, 2500); Order order2 = new Order(orderId + 2, tradingAcct, oil6MnthFuture, 1500, 2600); Order order3 = new Order(orderId + 3, tradingAcct, goldFuture, 200, 4300); Order order4 = new Order(orderId + 4, tradingAcct, goldFuture, 150, 4300); OrdersDAO orderDAO = new OrdersDAOInMemory(order1, order2, order3, order4); OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO); // when List<Order> orders = orderSearchService.getOrders(trader, tradingAcct, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }
  • 48. Specifying Inputs • Consider many non-trivial tests, with “raw” setup • Impact of changing one setter or constructor • Quantity of code there is to read, understand, change
  • 49. Specifying Inputs - Techniques • Helper methods • Object Mother Pattern • Test Data Builder Pattern
  • 50. @Test public void getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v1() { // given - inputs Trader trader = new Trader(); TradingAccount tradingAcct = new TradingAccount(new TradingFirm("TraderzRUs", "TRDRZ"), new ClearingFirm("TooBig2Fail", "2BIG2F"), ACTIVE); trader.setPermissions(Arrays.asList(new Permission(tradingAcct))); String goldIsinCode = "COF24680"; // given - system state Future oil3MnthFuture = new Future("OIL.3MNTH", new ISIN("OIL3M0123")); Future oil6MnthFuture = new Future("OIL.6MNTH", new ISIN("OIL6M0456")); Future goldFuture = new Future("GLD.3MNTH", new ISIN(goldIsinCode)); String orderId = "ordId"; Order order1 = new Order(orderId + 1, tradingAcct, oil3MnthFuture, 1000, 2500); Order order2 = new Order(orderId + 2, tradingAcct, oil6MnthFuture, 1500, 2600); Order order3 = new Order(orderId + 3, tradingAcct, goldFuture, 200, 4300); Order order4 = new Order(orderId + 4, tradingAcct, goldFuture, 150, 4300); OrdersDAO orderDAO = new OrdersDAOInMemory(order1, order2, order3, order4); OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO); // when List<Order> orders = orderSearchService.getOrders(trader, tradingAcct, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }
  • 51. @Test public void // Using helpers getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v2() { // given - inputs String accountCode = "TRDRZ"; String goldIsinCode = "GLD24680"; TradingAccount tradingAcct = createTradingAccount(accountCode); Trader trader = createTraderWithPermissionsFor(tradingAcct); // given - system state Future oil3MnthFuture = createFuture("OIL.3MNTH", "OIL3M0123"); Future oil6MnthFuture = createFuture("OIL.6MNTH", "OIL6M0456"); Future goldFuture = createFuture("GLD.3MNTH", goldIsinCode); Order order1 = createOrder(accountCode, oil3MnthFuture, qty(1000), price(2500)); Order order2 = createOrder(accountCode, oil6MnthFuture, qty(1500), price(2600)); Order order3 = createOrder(accountCode, goldFuture, qty(200), price(4300)); Order order4 = createOrder(accountCode, goldFuture, qty(150), price(4300)); OrderSearchService orderSearchService = createOrderService( createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = orderSearchService.getOrders(trader, accountCode, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }
  • 52. @Test public void // Using helpers + hiding irrelevant details getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", dummyFuture()), order2 = createOrder("TRDRZ", dummyFuture()), order3 = createOrder("TRDRZ", createFuture("GLD24680")), order4 = createOrder("TRDRZ", createFuture("GLD24680")); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }
  • 53. @Test public void getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v1() { // given - inputs Trader trader = new Trader(); TradingAccount tradingAcct = new TradingAccount(new TradingFirm("TraderzRUs", "TRDRZ"), new ClearingFirm("TooBig2Fail", "2BIG2F"), ACTIVE); trader.setPermissions(Arrays.asList(new Permission(tradingAcct))); String goldIsinCode = "COF24680"; // given - system state Future oil3MnthFuture = new Future("OIL.3MNTH", new ISIN("OIL3M0123")); Future oil6MnthFuture = new Future("OIL.6MNTH", new ISIN("OIL6M0456")); Future goldFuture = new Future("GLD.3MNTH", new ISIN(goldIsinCode)); String orderId = "ordId"; Order order1 = new Order(orderId + 1, tradingAcct, oil3MnthFuture, 1000, 2500); Order order2 = new Order(orderId + 2, tradingAcct, oil6MnthFuture, 1500, 2600); Order order3 = new Order(orderId + 3, tradingAcct, goldFuture, 200, 4300); Order order4 = new Order(orderId + 4, tradingAcct, goldFuture, 150, 4300); OrdersDAO orderDAO = new OrdersDAOInMemory(order1, order2, order3, order4); OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO); // when List<Order> orders = orderSearchService.getOrders(trader, tradingAcct, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }
  • 54. @Test public void // Using helpers getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v2() { // given - inputs String accountCode = "TRDRZ"; String goldIsinCode = "GLD24680"; TradingAccount tradingAcct = createTradingAccount(accountCode); Trader trader = createTraderWithPermissionsFor(tradingAcct); // given - system state Future oil3MnthFuture = createFuture("OIL.3MNTH", "OIL3M0123"); Future oil6MnthFuture = createFuture("OIL.6MNTH", "OIL6M0456"); Future goldFuture = createFuture("GLD.3MNTH", goldIsinCode); Order order1 = createOrder(accountCode, oil3MnthFuture, qty(1000), price(2500)); Order order2 = createOrder(accountCode, oil6MnthFuture, qty(1500), price(2600)); Order order3 = createOrder(accountCode, goldFuture, qty(200), price(4300)); Order order4 = createOrder(accountCode, goldFuture, qty(150), price(4300)); OrderSearchService orderSearchService = createOrderService( createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = orderSearchService.getOrders(trader, accountCode, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }
  • 55. @Test public void // Using helpers + hiding irrelevant details getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", dummyFuture()), order2 = createOrder("TRDRZ", dummyFuture()), order3 = createOrder("TRDRZ", createFuture("GLD24680")), order4 = createOrder("TRDRZ", createFuture("GLD24680")); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }
  • 56. Specifying Inputs - Techniques • Object Mother Pattern
  • 57. public class ObjectMother { ! public static OrdersDAOInMemory createOrdersDAO(Order order1, Order order2, Order order3, Order order4) { return new OrdersDAOInMemory(order1, order2, order3, order4); } ! public static OrderSearchService createOrderService(OrdersDAO ordersDAO) { OrderSearchService orderSearchService = new OrderSearchServiceImpl(ordersDAO); return orderSearchService; } private OrderSearchService prepareOrderService(Order order1, Order order2, Order order3, Order order4) { OrdersDAO orderDAO = createOrdersDAO(order1, order2, order3, order4); OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO); return orderSearchService; } ! public static Order createOrder(String accountCode, Future dummyFuture) { return createOrder(accountCode, dummyFuture, qty(0), price(0)); } ! public static Order createOrder(String accountCode, Future future, int qty, int price) { return new Order(nextOrderId(), accountCode, future, qty, price); } private Order createOrder(String orderId, TradingAccount tradingAcct, Future future, int qty, int price) { return new Order(orderId, tradingAcct.getTradingFirm().getCode(), future, qty, price); } ! public static Future createFuture(String desc, String isinCode) { return new Future(desc, new ISIN(isinCode)); } public static Future createFuture(String isinCode) { return createFuture(DEFAULT_FUTURE_DESC, isinCode); } ! public static Future dummyFuture() { return createFuture(DEFAULT_FUTURE_DESC, DEFAULT_ISIN_CODE); } !
  • 58. Specifying Inputs - Techniques • Test Data Builder Pattern
  • 59. @Test public void // Using Test Data Builders getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = aTrader().with(aPermission() .with(aTradingAccount() .with(aTradingFirm().withCode("TRDRZ")))).build(); // given - system state OrderBuilder matchingOrder = anOrder().withAccountCode("TRDRZ") .with(aFuture().with(anISIN().withIsinCode("GLD24680")).build()); Order order1 = anOrder().withAccountCode("TRDRZ").build(), order2 = anOrder().withAccountCode("TRDRZ").build(), order3 = matchingOrder.build(), order4 = matchingOrder.build(); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }
  • 60. public class TraderBuilder { public static String DEFAULT_USERNAME = "trader1"; public static UserDetail DEFAULT_USERDETAIL = new UserDetail(); public static List<Permission> NO_PERMISSIONS = Collections.emptyList(); public static List<Restriction> NO_RESTRICTIONS = Collections.emptyList(); ! private String userName = DEFAULT_USERNAME; private UserDetail userDetail = DEFAULT_USERDETAIL; private List<Permission> permissions = NO_PERMISSIONS; private List<Restriction> restrictions = NO_RESTRICTIONS; private TraderBuilder() { } public static TraderBuilder aTrader() { return new TraderBuilder(); } public Trader build() { Trader trader = new Trader(); trader.setUserName(userName); trader.setUserDetail(userDetail); trader.setPermissions(permissions); trader.setRestrictions(restrictions); return trader; } public TraderBuilder withUserName(String userName) { this.userName = userName; return this; } ! public TraderBuilder with(UserDetail userDetail) { this.userDetail = userDetail; return this; } public TraderBuilder withPermissions(List<Permission> permissions) { this.permissions = new ArrayList<>(permissions); return this; } public TraderBuilder this.permissions return this; } public TraderBuilder this.permissions return this; } with(PermissionBuilder permission) { = new ArrayList<>(Arrays.asList(permission.build())); withNoPermissions() { = Collections.emptyList(); ! public TraderBuilder with(RestrictionBuilder restriction) { this.restrictions = new ArrayList<>(Arrays.asList(restriction.build())); return this; } public TraderBuilder withRestrictions(List<Restriction> restrictions) { this.restrictions = new ArrayList<>(restrictions); return this; } public TraderBuilder but() { return new TraderBuilder() .withUserName(userName) .with(userDetail) .withPermissions(permissions) .withRestrictions(restrictions); } }
  • 61. @Test public void // Using Test Data Builders getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = aTrader().with(aPermission() .with(aTradingAccount() .with(aTradingFirm().withCode("TRDRZ")))).build(); // given - system state OrderBuilder matchingOrder = anOrder().withAccountCode("TRDRZ") .with(aFuture().with(anISIN().withIsinCode("GLD24680")).build()); Order order1 = anOrder().withAccountCode("TRDRZ").build(), order2 = anOrder().withAccountCode("TRDRZ").build(), order3 = matchingOrder.build(), order4 = matchingOrder.build(); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }
  • 62. @Test public void // Using helpers + hiding irrelevant details getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", dummyFuture()), order2 = createOrder("TRDRZ", dummyFuture()), order3 = createOrder("TRDRZ", createFuture("GLD24680")), order4 = createOrder("TRDRZ", createFuture("GLD24680")); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }
  • 63. @Test public void // Using Test Data Builders to create input data variations getOrders_GivenTraderWithVariousPermissions_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v5() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; TradingAccountBuilder tradingAccount = aTradingAccount().with(aTradingFirm().withCode("TRDRZ")); Trader traderWithNoPerms = aTrader().withNoPermissions().build(); Trader traderWithPerms = aTrader().with(aPermission().with(tradingAccount)).build(); Trader traderWithInactivePerms = aTrader().with(aPermission().with( tradingAccount.but().isInActive())).build(); // given - system state OrderBuilder matchingOrder = anOrder().withAccountCode("TRDRZ") .with(aFuture().with(anISIN().withIsinCode("GLD24680")).build()); Order order1 = anOrder().withAccountCode("TRDRZ").build(), order2 = anOrder().withAccountCode("TRDRZ").build(), order3 = matchingOrder.build(), order4 = matchingOrder.build(); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders1 = service.getOrders(traderWithNoPerms, searchAccount, searchIsin); List<Order> orders2 = service.getOrders(traderWithPerms, searchAccount, searchIsin); List<Order> orders3 = service.getOrders(traderWithInactivePerms, searchAccount, searchIsin); // then assertThat(orders1).isEmpty(); assertThat(orders2).containsOnly(order3, order4); assertThat(orders3).isEmpty(); }
  • 64. Specifying Inputs - Techniques • Test Data Builder Pattern - with tool support • Make-It-Easy - addresses Builder class boiler-plate • Model Citizen - annotation-based “Blueprints”
  • 65. Specifying Inputs - Techniques • Test Data Builder Pattern - Advantages • Your test / spec documents only the inputs that matter • Decouples your test from being impacted by a wide range of changes • Supports “declarative” code style + useful templates • user = EXPIRED_USER.but( with ( aBalance, 100));
  • 66. Specifying Inputs - Techniques • Test Data Builder Pattern - Disadvantages • Requires much new test-supporting code • Chained methods take getting used to
  • 67. Specifying the System State • State of the SUT / the Test Context • Communicating what the context, limitations and boundaries are • Isolating your test from certain dependencies / interactions • Use of Test Doubles • Hand-rolled Test Double • common misnomer Mocking Frameworks
  • 68. Specifying the System State • Problems + solutions for Specifying Inputs normally apply • Unique to System State is your Test Isolation requirements
  • 69. Specifying the System State - Techniques • Hand-rolling Test Doubles • Using Test Isolation frameworks to provide Test Doubles • “Mocking” + “Mocks Aren’t Stubs”
 http://martinfowler.com/articles/mocksArentStubs.html • Dummy / Fake / Stubs / Mocks / Spies
  • 70. Specifying the System State - Techniques • Stub being used in a state-based test
  • 71. @Test public void // Stubbing system state (Mockito) + using helpers + hiding irrelevant details getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v2() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); OrderSearchService service = createOrderService(ordersDAOStub); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order1, order2); }
  • 72. @Test public void // Stubbing system state + stubbing inputs (Mockito) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; // Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); Trader traderStub = mock(Trader.class); Permission permissionStub = mock(Permission.class); given(traderStub.getPermissions()).willReturn(Arrays.asList(permissionStub)); TradingAccount tradingAccountStub = mock(TradingAccount.class); given(permissionStub.getTradingAccount()).willReturn(tradingAccountStub); given(tradingAccountStub.isActive()).willReturn(Boolean.TRUE); given(tradingAccountStub.getTradingFirm()).willReturn(new TradingFirm("", searchAccount)); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); OrderSearchService service = createOrderService(ordersDAOStub); // when List<Order> orders = service.getOrders(traderStub, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order1, order2); }
  • 73. @Test public void // Stubbing system state + stubbing input (Mockito w/ Deep Stubs) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3_1() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; // Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); Trader traderStub = mock(Trader.class); Permission permissionStub = mock(Permission.class, RETURNS_DEEP_STUBS); given(traderStub.getPermissions()).willReturn(Arrays.asList(permissionStub)); given(permissionStub.getTradingAccount().isActive()).willReturn(Boolean.TRUE); given(permissionStub.getTradingAccount().getTradingFirm().getCode()).willReturn(searchAccount); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); OrderSearchService service = createOrderService(ordersDAOStub); // when List<Order> orders = service.getOrders(traderStub, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order1, order2); }
  • 74. Specifying the System State - Techniques • A Mock being used in a interaction-based test
  • 75. @Test public void // Using a Mock to test interactions + Stubbing system state (Mockito) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4_1() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); PermissionService permissionServiceMock = mock(PermissionService.class); OrderSearchService service = createOrderServiceV2(ordersDAOStub, permissionServiceMock); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then verify(permissionServiceMock).hasViewPermissions(trader, order1); verify(permissionServiceMock).hasViewPermissions(trader, order2); }
  • 76. @Test public void // Using a Mock to test interactions + Stubbing system state (Mockito) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4_2() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); PermissionService permissionServiceMock = mock(PermissionService.class); given(permissionServiceMock.hasViewPermissions( any(Trader.class), any(Order.class))).willReturn(Boolean.TRUE); OrderSearchService service = createOrderServiceV2(ordersDAOStub, permissionServiceMock); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then verify(permissionServiceMock).hasViewPermissions(trader, order1); verify(permissionServiceMock).hasViewPermissions(trader, order2); assertThat(orders).containsOnly(order1, order2); }
  • 77. Specifying the System State - Techniques • Often a test will have • Zero to many Stubs. Zero or one Mock • Mocks help you answer questions about what happened • When a Mock is present, you ask it something at the end • A Stub should not be asserted against • There to help create + control the System State for the test
  • 78. Specifying the Event • Often trivial • Might be a sequence of events that we want to fire • Sequence might actually be context / SUT setup • Practices? Tools?
  • 79. Specifying it All Together • Spring MVC Test support / DSL @Test public void helloWorld() throws Exception { mockMvc.perform(get("/hello").accept(MediaType.TEXT_PLAIN)) .andDo(print()) // print the request/response in the console .andExpect(status().isOk()) .andExpect(content().contentType(MediaType.TEXT_PLAIN)) .andExpect(content().string("Hello World!")); }
  • 80. Principles, Practices + Tool Support • Structural Principles / Characteristics • Qualities of a “good” test / spec • Practices + Tools for improving test qualities
  • 81. Continuous Testing • Continuous Integration - development practice • CI tools - run automated tests frequently • Continuous Testing - development practice • CT tools - run automated tests locally, frequently • e.g. Infinitest, JUnitMax
  • 82. Overview • Automating • Testing Categories + Scope • Development Testing • Principles, Practices + Tool Support
  • 83. Summary • Tools/APIs just help with heavy lifting • Sound coding principles are paramount • Creating unreadable and unmaintainable test, even with “the right” tools • Unreadable + high maintenance tests will hinder efficiency
  • 84. Summary • The power of marginal improvements • for … each, <> operator, closure support, Guava • Continuous iterative improvements nudge us forward • Continuous iterative improvements push down cost of reading, writing, evolving tests • Drives the cost down to “mass market” level • Beyond Java
  • 85. Resources • Books • GOOS - Growing Object Oriented Software, Guided By Tests • Effective Unit Testing • Working Effectively with Legacy Code • xUnit Test Patterns - Refactoring Test Code
  • 86. Resources • Podcasts • • Hanselminutes - Roy Asherove - The Art of Unit Testing • • Sustainable Test Driven Development (Series) Hanselminutes - Quetzal Bradley - Beyond Unit Testing All Code Examples are available at • http://github.com/cathalking/devtestevolution